A Guide to 
Collaborative Research 


10T1 


HIV Preventi 


In 


Working Together 
A Guide to Collaborative 


Research in HIV Prevention 


af 


- For additional copies of this manual, please see: 
WWW.CAPS.UCSF.EDU/COLLABORATION. 


Center for AIDS Prevention Studies 
AIDS Research Institute 

University of California San Francisco 
74 New Montgomery, Suite 600 

San Francisco, CA 94105-3444 


www.caps.ucsf.edu 

© September 2001, University of California. Any part of this manual 
may be copied or adapted without permission; however, copies may 
not be sold, and the University of California San Francisco should be 
cited as the source. 


Funded by the National Institutes of Mental Health. 


€9 Printed on recycled paper. 


WAS "3S 
103994 “226 


PREFACE 


CHAPTER 1 


CASE STUDY 


CHAPTER 2 


CHAPTER 3 


CASE STUDY 


CHAPTER 4 


CHAPTER 5 


CASE STUDY 


CHAPTER 6 


CHAPTER 7 


CASE STUDY 


APPENDIX A 


APPENDIX B 


APPENDIX C_ 


APPENDIX D 


APPENDIX E 


CONTENTS 


Vere tise tins Manwdl -..s4 s+ cas heer ote + she oe me oe teeters At 1 
The STOP AIDS Project’s Collaboration with CAPS by Dan Wohlfeiler . . 5 
Ruin Collaborate! <select 7 
Garting a Collaboration... -- +--+ -2--+2 = pe ne ere 15 


Against All Odds: Conducting Research in a CBO by Drew Feraios .... 23 


Developing a Research Question ....--.----++++-+7 terrestres: 25 
Collaborative Data Collection and Analysis ....----++-+++++++++7** 31 
Working with Youth in HIV Prevention Research by Gary Harper ....--. 41 
Incorporating Findings into Research and Service ......+---+++++++> 43 
Disseminating What You've Leatmed .-- 5-2. ae 49 
New Conservatory Theater Center by Lisa ett ose eee recs 55 
Reverenices and Selected eadimgs) © ---5 . c= 5 a «ff 
The CAPS/NCG Collaborative Research Consortium Model .....---- 59 
Finding Collaborative Research Partners. ues esas ees a eet 61 
Glostaey Of Research LermS 262-6 1t see tte teens tery ese 62 
“CofA tec) ROIs a a aR 64 


Contents Page Wl 


PREFACE 


This manual is a result of 15 years of research and experience and was created to encourage 
and support community collaborative research. Academic researchers at the Center for 
AIDS Prevention Studies (CAPS), AIDS Research Institute, University of California San 
Francisco, have been conducting research in collaboration with Community-Based 
Organization (CBO) partners since 1986. In 1991, the Technology and Information 
Exchange Core (TIE Core) at CAPS initiated a more structured, deliberate strategy for 
linking researchers and CBOs to conduct community-relevant research. Five CBOs were 
matched with 5 CAPS researchers and funded with small evaluation grants for two years. 
In 1994, Northern California Grantmakers (NCG) — AIDS Task Force and CAPS began 
a partnership which would continue for the next 6 years and fund 18 CBO/CAPS research 
projects in two collaborative consortia. The CAPS/NCG Collaborative Research 
Consortium Model has been evaluated and found effective, and subsequently been replicated 
with various modifications across California, and in New York and Chicago. Please see 


Appendix B for a detailed description of the CAPS/NCG Model. 


After 8 years of conducting Collaborative Research Consortia, the TIE Core began to 
research the methods of collaboration in an effort to understand what elements made for 
successful collaborations. The Legacy Project looked at best practices for collaboration and 
capacity building among the 18 CAPS/NGG partnerships. The Collaborative Data Analysis 
Project (CDAP) identified and described models of effective collaboration during the data 
analysis process. Two outside evaluators, Harder & Company Community Research and 
Tom Backer & Associates, also looked at the effectiveness of the CAPS/NCG Collaborative 


Research Consortium Model. 


This manual has been written and reviewed by both researchers and CBO staff. We would 
like to thank the following CBO partners for participating in the Collaborative Research 
Consortia and the research projects: AIDS Project of the East Bay; AIDS Resources, 
Information and Services of Santa Clara County (ARIS); Coalition of Immigrant & 
Refugee Rights & Services (CIRRS); East Bay Community Recovery Project; Face To 
Face/Sonoma County AIDS Network; La Clinica de la Raza-Fruitvale Health Project, Inc.; 
Larkin Street Youth Center; Centerforce, Health Programs Division; Marin AIDS Project; 
Mid-Peninsula YWCA; New Conservatory Theater Center; STOP AIDS Project; 
Tenderloin AIDS Resource Center (TARC); Tri-City Health Centers and Huckleberry 
Youth Programs (formerly Youth Advocates, Inc.). We would like to thank the following 
researcher partners for participating in the Collaborative Research Consortia and the 
research projects: Michael Acree, Diane Binson, Don Chambers, Katherine Haynes Sanstad, 
Kyung Hee Choi, Michael De Mayo, Rafael Diaz, Maria Ekstrand, Bonnie Faigeles, Ellen 
Goldstein, Cynthia Gomez, Olga Grinstead, Tom Hall, Gary Harper, Jay Paul, Kathleen 
Quirk, Fabio Sabogal, Ron Stall, Ariane van der Straten and Bill Woods. We would like 
also to thank the following funding partners from Northern California Grantmakers — 
AIDS Task Force for participating in the Collaborative Research Consortia: John Mortimer, 
Wendy Everett, Ruth Brousseau and Nancy Frank. 
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We would also like to thank the following reviewers for this manual: Lisa Heft, Blanca Ortiz, 
Bill Snow, Dan Wohlfeiler, David Seal and Karen Vernon. 


Diane Binson, Pamela DeCarlo, Bonnie Faigeles, Beth Freedman, Ellen Goldstein, 

Olga Grinstead, Kathleen McCartney, Scott Stumbo, Dan Wohlfeiler and Bill Woods are 
responsible for writing and editing this manual. The research team for Legacy and CDAP is: 
Diane Binson, Maria Ekstrand, Bonnie Faigeles, Beth Freedman, Sandi Galvez, Ellen 
Goldstein, Colleen Hoff, Ann Richards, Karen Vernon, Bill Woods. 
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Chapter 


How 


to Use This Manual 


Our hope is that this manual will inspire you to try collaborative research, 


provide you with information to make that a successful experience if you 


do try it and enhance any collaborations in which you are already engaged. 


Chapter 


% 


major goal for both researchers and CBO staff is to make a difference in people’s 

lives by helping prevent new HIV infections. After 20 years, there is a large body 

of scientific research as well as front line experience about what works in HIV 
prevention. Collaboration uses both these resources to ensure that we’re doing our best to 
achieve that goal. 


TOP FIVE REASONS WHY YOU MIGHT HAVE PICKED UP THIS MANUAL 

1. You have some questions about the effectiveness of your agency’s programs. Or you have 
a question about a certain population and you’d like to know more. You know that the 
best way to get answers would be to link up with a researcher or CBO, and you’d like the 
process to be as inclusive and collaborative as possible. 


2. You wrote in your grant proposal that you would work collaboratively with a researcher or 
CBO partner. Now the grant is funded and you need some suggestions about how to start. 


3. Your local Health Department sent you this manual and you think this means they’re 
going to expect you to collaborate with a researcher. You want some quick guidance. 


4. Your local Community Planning Group (CPG) has announced that all funded _¢ 
projects must incorporate research into their design and evaluation. yy 


5. You’ve been working in what is supposed to be a collaboration between 
researcher and CBO staff, but it’s not going as well as you'd like. 


Whatever the reason, in these pages you'll find a wealth of information 
and guidance drawn from research and years of experience with collaboration. 
In this manual we will discuss the process of collaborative research from concep- 
tualizing a research question to analyzing data and disseminating research findings. Our 
hope is that this manual will inspire you to try collaborative research, provide you with infor- 
mation to make that a successful experience if you do try it and enhance any collaborations 
in which you are already engaged. 


WHAT IS COLLABORATIVE RESEARCH? 

For the purposes of this manual, collaborative research is research that is conceived, planned 
and conducted in collaboration between academic researchers and CBOs. It also includes 
data analysis and interpretation and dissemination of findings done in collaboration. 
Research may be defined as collaborative f a collaborative process was employed in any of 
these stages of research; the more stages in which collaboration can be accomplished, 
however, the more potential benefits accrue. Our perspective is that collaborative research 
often results in better programs and more relevant findings compared to cases when research 
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is conducted either by a researcher alone without community input or by an agency alone 
without the collaboration of a researcher. : 

The goal of collaborative research is to bring the experience, observations and skills of 
community service providers together with the technical expertise of researchers to produce 
meaningful community-based research questions which enhance research methods and 
modify and develop community programs. 

Although our work has focused on the application of collaborative research to HIV 
prevention program evaluation, we hope that these principles will be useful to promote and 
guide collaboration in other areas as well. We also encourage those who are interested in 
learning more about collaborative research to seek out examples and lessons learned from 
other areas of public health including smoking cessation, violence prevention, family planning 


and breast cancer research. 


KEY POINTS: COLLABORATIVE RESEARCH 


These are the “golden rules” of conducting research collaboratively. Listed at the end of 
each chapter in this manual are key points for collaboration, and the lessons below will be 
repeated in many of them. 


Get funding. Set appropriate expectations for research efforts and fund accordingly; 
budget adequate resources for the project. 


Specify roles. Staff the research project appropriately and specify key roles for researchers, 
CBO staff, funders, and administrative support. A clear understanding 
of responsibility and task assignment is essential. 


Be committed. Commitment from all parties of the collaboration is essential; buy-in from 
all levels of staff at both organizations is critical. Researchers should commit to active 


engagement with the CBO. The CBO should have a commitment to the research question. 


Provide training. Acknowledge both CBO and academic researcher expertise and 
experience. Offer appropriate skills training when needed. 


Meet regularly. Regular meetings between CBO staff and researcher staff are an 
effective way to foster peer support and sustain enthusiasm and commitment to the process. 


Be flexible. Chaos happens. Be flexible and acknowledge real life at a CBO and the 
potential bureaucratic delays of conducting research. 


Start early. The more time allowed for setting up a collaboration, the better prepared each 
party will be. 
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CASE STUDY 


THE STOP AIDS PROJECT’S COLLABORATION WITH CAPS 
Dan Wohlfeiler (Education Director, 1990-98) 


e at the STOP AIDS Project were particularly lucky to 
have access to academic researchers with a strong 


commitment to the community. While many academics 


represent an excellent example of a “hard to reach” population, 
Center for AIDS Prevention Studies (CAPS) researchers both studied 


and were members of our own community. It was simply a matter of 


calling them up and asking for their help. 


We wanted to measure the impact of our 
interventions on our workshop participants. 
Over many bagel breakfasts with CAPS’ Ron 
Stall and Maria Ekstrand, we developed a six- 
month follow-up questionnaire. We quickly 
learned that our own system lent itself well 
to sending out initial and even follow-up 
questionnaires, but getting those follow-up 
questionnaires back proved to be a real chal- 
lenge — and took a lot of effort to integrate 
into our own program. While we didn’t have 
a control group to be able to draw real 
conclusions, our own “reckoning day” 
showed us that we were moving participants 
in the right direction (much to our great 
relief). The data boosted our staff and volun- 
teers’ morale, and kept our own fundraiser 


very, very happy. 


- Later on, we were able to collaborate with 
CAPS and other researchers to see what our 
project’s impact was on the community. We 
worked with the Young Men’s Health Study, 
which surveyed a group of randomly-sampled 
men and gave them HIV tests each year. In 
their survey, we added a lot of questions 


Case Study: The STOP AIDS Project's Collaboration with CAPS 


about what their exposure to or involvement 
in the Project had been in order to find out if 
we had any impact on their seroconversions. 


When we were writing our theoretical 

justification, our researcher said, “This year . 
Social Learning Theory is what the funders 

are looking for.” | 
told him that | 
could describe our 
project using one of 
any number of theo- 
ties, from Social 
Learning Theory to 
the Health Belief 
Model to Stages of 
Change to theories 
of social support and 
community organiz- 
ing, since in real life 


Two men ata 
STOP AIDS safer sex 


educational meeting. 


they all explained 
parts of our project. 


Our results were modest, and we didn’t have 


the degree of impact we wanted. We did learn 
that we were reaching a very high percentage 
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of our population with our outreach, media, 
and group workshops. | wish I could tell 
you the final results — but like so many 
researchers, we have found that it’s a lot 
easier to gather the data than to analyze 
them fully, write them up and publish them. 
We also proved that CBOs can, in fact, 
conduct outcome evaluation if they can 
partner with the right researchers. 


Our Project had always recruited men off the 
street into workshops with other men who 
didn’t know one another. Based on social 
support theories — which suggest that if 
people have explicit practical and emotional 
support for healthy behavior they’re less likely 
to take risks — I wanted to try recruiting 
whole social groups, from the gay chorus and 
gay softball league to groups of friends, 
for our workshops. I hoped that instead of 
sending men out from the workshops back 
into the community where they may never 
run into each other, these social groups 
would allow the explicit conversations 
which started in the workshops to continue. 


Recruitment turned out to be a much greater 


challenge than we ever imagined — and 
much to my own disappointment, we ended 
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up abandoning the project halfway through 
our funded collaboration with CAPS. We did 
get some great qualitative information about 
how men in groups did and did not ask one 
another about their risk behavior, and how 
they used groups as a way to get a life outside 
of AIDS. So all was not lost. Could more 
formative evaluation have helped us under- 
stand better how hard this would be to carry 
out? Probably. And despite the fact that I’m 
no longer at STOP AIDS, the funding for 
that intervention has long-since dried up, 
and the staff I asked to implement swear 
they’ll never attempt it again, I still remain 
convinced that groups offer a real advantage 
for HIV prevention. Although it goes 
against my belief that if it didn’t work, don’t 
repeat it, I still have faith that a community 
member and researcher will figure out how 
to improve our design and see if it really can 
make a difference. 


Case Study: The STOP AIDS Project’s Collaboration with CAPS 


Chapter 


Why Collaborate? 


Researchers and CBO staff working collaboratively can enhance the strengths 


of both fields, and enrich the research process by 
bringing various skills and knowledge together. 


Chapter 


4 


na perfect world, academic researchers, CBO staff, health departments, and AIDS 

activists would all work together. We would recognize the experience and wisdom 

that each had to offer in our collective efforts to stop HIV from spreading. We would 
interact regularly, share our knowledge and expertise, and use all the lessons learned to 
develop better prevention programs. 

Unfortunately, when conducting research, designing programs or developing theories, 
most CBO staff and researchers do not automatically think of turning to each other for 
input. This is understandable, given that collaboration is not something either partner is 
necessarily trained to do or has the funding for. There is also a long history of mistrust and 
stereotyping from both sides. One way to understand the disconnect between CBOs who 
conduct HIV prevention programs and scientists who conduct HIV prevention research is 
to think of these as two separate cultures. The interactions between the two represent the 
equivalent of a cross-cultural experience. As such, we can expect misunderstanding and 
culture shock between the two worlds. But, as in all cross-cultural communications, we can 
also expect learning, expanded perspectives, and excitement when we meet successfully. 


COLLABORATION ASA CROSS-CULTURAL EXPERIENCE 
The worlds of research and CBO program delivery are often separate, with each having its 
own sense of time, incentives, language, resources and authorities, among other things. 


© Time. An immediate client crisis that needs to be handled right now vs. a five-year 
research grant and submitting findings to a journal that won’t publish an article 


for 6-12 months 


¢ Incentives. Units of service, number of unduplicated clients 
vs. numbers of grants and publications 


e Jargon and language. CTRPN and HE/RR vs. random 


sample and statistical significance 


e Resources. Peer networks, trainings vs. research, 
administrative support, Internet, library 


° Authorities. Clients, funders, departments of health, 
experience, colleagues vs. scientific literature 


When people who work with and identify with one of these cultures interact with some- 
one from the other, it’s not hard to see why the result can be often unsatisfying or confusing. 
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OVERCOMING CULTURAL BARRIERS 


If we consider research and program delivery as two different cultures, we realize that we 
cannot automatically assume that we understand the meaning of behavior or how norms 


in the other culture operate. 


It is always important to ask questions 

If a researcher asks for CBO feedback on an instrument by next week and is told that the 
CBO staff can’t get to it, she shouldn’t assume that the CBO staff are indifferent or sending 
the message that the research isn’t a priority. Perhaps the CBO is having a site visit or one 
of the key staff members just went out on disability. Or perhaps the computer system 1s 
down and they don’t have the budget to fix it. . 

If a CBO proposes a research question for joint study and the researcher is unenthusiastic, 
it may be because the resources can’t answer a question of that complexity or because a 
paper just came out in a journal that answers the same question. From both sides, it’s 
important to ask for clarification. 

If the researcher wants to review and modify codes on a questionnaire or a CBO staff 
member wants to revise the same questionnaire for cultural appropriateness, the common 
goal is to develop a questionnaire that will best give voice to community experiences. If a 
researcher focuses on publishing in an academic journal and a CBO program manager 
spends time on a presentation to the local Community Planning Group, both are engaged 
in efforts to disseminate their findings. They just have different resources and expectations 
about the accepted or best way to do it. Neither one or the other is better; combining both 
may be the best solution. 

Cross-cultural differences do impact collaborative relationships, but in real life, the line 
between researcher and CBO staff is often blurred. CBOs often have staff with graduate 
degrees and research skills — people who look at the intake forms, the client satisfaction 
surveys, and the workshop evaluations as data, feeding the information back to the program 
for further development. Also, researchers are members of the HIV-infected and affected 
community. For example, after his input was dismissed at a Community Planning Meeting 
because he was presenting epidemiological data, an HIV+ researcher asked, “How many of 
my friends do I need to bury, and how sick do I need to be before I am considered part of 
“The Community?” The bottom line is that we are all working for the same goal, all commit- 
ting our time and skills to ending the epidemic. We can do it better when we work together. 


DISADVANTAGES OF WORKING SEPARATELY 

Researchers and CBOs can and do work separately on research projects. Researchers who 
want to design or evaluate an HIV prevention intervention could (and often do) develop 
the intervention based on theoretical constructs, recruit participants on their own and 
conduct the intervention in venues created specifically for that research study. Researchers 
could (and often do) collect, enter, analyze and interpret data and disseminate their findings 
without community input. There are several disadvantages to this approach: 


¢ Researchers often have the money and time to design interventions that are so 
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comprehensive and resource-intensive that they are impossible to replicate in a 
community setting. 


© Researcher-driven studies may be based on what is printed in scientific journals rather 
than what is most currently needed among clients at risk for HIV. 
7 


e Research findings may be interpreted based on the numeric data or one-time, focused 
interviews, with little knowledge of the daily life of clients surveyed. 


On the other side, CBO staff who want to design or evaluate an HIV prevention 
program could (and often do) develop the intervention based on what they’ve seen and 
heard from their clients, offer the intervention to the “usual suspects” in the community 
and design or adapt pre- and post-test surveys for evaluation. In fact, CBO staff have been 
pushed in this direction as more and more funders are demanding the development and 
completion of a program evaluation plan. For CBOs, working alone has its disadvantages: 


© Without consulting the HIV prevention research literature, CBO staff may duplicate 
past failed efforts. Certain program components may have been proven unsuccessful, 
while others have proven to be highly effective. 


© CBO staff may be conducting research, especially evaluation research, only to fulfill 
requirements from the funder. Often CBO staff don’t have the time or inclination to 

~ step back and look at the big picture, or to make sure the research they’re conducting 
helps the agency as well as the funder. 


© CBO staff may rely on anecdotal measures of success (e.g., clients really like the 
program). Without evaluation research, it is often difficult to know if programs actually 
help change behaviors or attitudes of clients. 


ADVANTAGES OF COLLABORATION 
e Programs designed or evaluated in collaboration with CBOs are already known to be 


feasible in community settings. 


© Collaborating with a CBO ensures that research actually reflects the current needs of 
affected communities. 


© CBO staff bring valuable insights into why clients may be responding in a certain 
way or making certain choices, which results in richer and more useful data analysis 


and interpretation. 


e After 20 years of scientific research on HIV/AIDS there is a large body of theoretical 
and applied knowledge about what works in HIV prevention. Researchers are often 
well versed in this literature and can assist CBO staff in adapting programs which have 


already been tested and proven effective. 


¢ Researchers have the technical expertise to hone a general question into a specific and 
well-defined research question and to develop a research plan using the methods best 


suited to answer that question. 
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Researchers and CBO staff working collaboratively can enhance the strengths of both 
fields, and enrich the research process by bringing various skills and knowledge together. 
In addition, working collaboratively can build capacity for CBO staff and researchers: 


© Increase research skills of CBO staff through trainings. Increase skills of researcher 


for translating theory into programs. 


¢ Help secure funds (researcher can fund CBOs through 
erants, researchers can get money from Community 


Planning Groups, etc.). 


¢ Both parties will learn more about what makes an 


T 
effective program. : FIELD NOTES 


e Researchers can gain access to unique at-risk populations. A CBO providing services to 


¢ Both gain access to alternate sources of information and incarcerated men and their 


dissemination: CBOs to academic journals, researchers 
to newsletters and outreach workers. 


families knew they were doing 
innovative work but did not have 


the technical expertise to conduct 


IS COLLABORATION FOR YOU? 
Collaboration is not for everyone. Neither is a collabora- 


evaluation. An academic researcher 
knew that the incarcerated were an 


tive relationship necessary for every research project. 

There are times when a researcher might use a Community 
Advisory Board to solicit input and advice from CBO staff 
and community members about their project. CBO staff 
may prefer to hire a researcher consultant to conduct a 
needs assessment for their client population. As you'll 
_ see in the next chapters, research collaboration takes 
time and commitment. While we believe that 
collaborations can improve both research and 

services, not everyone has the time and resources 

to engage in collaborative research. 

There are also several different forms of collab- 
oration. Collaborative relationships can occur 
between CBO staff and researchers alone, between 
both partners with an outside facilitator, and as 
part of a consortium of other CBO/researcher 
teams. All of these can work and be useful, but 
it’s up to the partners to decide how they might 
work best. It should go without saying that in 
each case, respect and admiration: for each 
other’s strengths is first and foremost. In all 


important population to study but 
knew there were major obstacles to 


getting access to prisons and jails. 


The CBO director approached the 
researcher to provide technical assistance 
in evaluating their HIV prevention programs 
for incarcerated men. Each partner got what 
they wanted: evaluation expertise and an 


inroad to incarcerated populations. 


This collaboration has continued now for more 


than 8 years and has included the evaluation of 
/ five different onvivanes as well as the collabora- 
tive development of four interventions for inmates 
and their partners. Programs based on these inter- 
ventions are now being implemented throughout 


of these cases, researchers and CBO staff 


can work together on defining the research Te: CRENOET SPER ar ATIRGr ot Sena 
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question, developing the surveys, collecting the data, analyzing the data and disseminating 
the findings. 

When CBO staff and researchers collaborate on their own, there is often a lead agency, 
and this agency is most likely the one that controls the funding. In this case, one agency 
might make overall decisioms about what they want to study and the timeline and desired 
outcomes. Collaboration still occurs during the process of research, and the lead agency 
needs to be flexible and agree to compromise when needed. 

CRO staff and researchers can also work with a facilitator for their collaboration. A 
facilitator can work with a collaborative relationship from the beginning, overseeing and 
managing the collaboration, or can be called in at different times to help overcome obstacles. 
The facilitator can be a mediator and a trainer, helping form an even playing field between 
researcher and CBO staff, and providing ground rules as well as flexibility. 

Sometimes collaborative research is conducted with a consortium, where one lead agency 
oversees and manages several collaborative projects in a certain geographic area or in a 
certain field. This model is often used in health department or university settings where 
several programs may be funded from one source. Consortia usually have a lead facilitator 
trained in collaboration who convenes regularly scheduled meetings of all the agencies 
involved. In this model, collaborative projects can learn from each other and help problem 
solve as a group. Please see Appendix B for a description of the evaluated CAPS/NCG 
Collaborative Research Consortium Model. 


KEY POINTS: WHY COLLABORATE? 


Collaborative research is a cross-cultural experience. 


Ask questions. Don’t assume you know what the other side is thinking. 


There are advantages to collaborating for both researchers and CBO staff. 


Collaborative research isn’t for everyone or every situation. 
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Chapter 


Star ting a Collaboration 


Building a good collaborative relationship at the beginning will smooth the way 


for better research and better services. 


Chapter 


e 
ollaborations, like any relationship, require effort to get off the ground. Consider 
starting a collaborative research project with the same care used to start a new 
romantic relationship, find a caregiver for a child or choose a healthcare provider. 

Because collaborations between researchers and CBO staff are cross-cultural relationships, 

they require even more care and nurturing. This chapter is a step-by-step guide to conducting 

healthy collaborative research partnerships between researchers and CBO staff. We'll begin 
as the collaboration starts, then move into how to maintain the relationship. 


GETTING THE RIGHT FIT 
You have the time, commitment and desire to collaborate on a research project. Now, how 
do you find a partner? This is the beginning of a new relationship, and it’s important 
to start off right. Please see Appendix C for resources for finding collaborative 
research partners. 

Once you have potential names, call them and set up a time for 
an interview or “first date.” Ask for a resume and references and then 
actually call the references. Some things to look for and ask about are: 


e A history of conducting research. What kind of research has been 
happening in the organization? Does your potential partner know 
basically what to expect from a research project! 


ene = ete 


° A history of collaboration. Has the person or organization collaborated 
with others before? How did the experience go? 


¢ Knowledge/experience with the population. How long have they been working in this 
area? Are they aware of the basic needs and history of the population? 


° Specific requirements of the project. Does the researcher or agency staff have the 
specific skills needed for this project? For example, does the researcher need to 
speak/understand Spanish? Does the CBO need to have access to African-American 


men who have sex with men? 


If at all possible, interview several possible partners to be able to compare strengths and 
weaknesses. It may be the case that there is only one CBO or only one researcher working 
in HIV prevention in your area. Even if you don’t have the luxury of choosing between 
several candidates, it is still important to have an initial interview and ask questions. This 
lets the partner know that collaboration will be taken seriously, and allows time to get to 
know a bit more about each other's background. It is also an opportunity to educate potential 


partners about what to expect. 
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ARE YOU COMMITTED? | 
Once both the CBO staff and researcher are clear about their goals for collaborating on a 


research project, it’s time to check that the commitment to work through this process is 
there. The primary areas to check are time, support and funding. 


© Time. Does each partner’s job allow for the work required in collaborative research? 
Assume a research project will take at least 10-20% of your time for one or two years 
(time spent each month will vary depending on the stage of research). 


¢ Support. Does each partner’s agency and direct supervisor support a research collabora- 
tion? Are the support staff (statistician, outreach workers, project assistants) aware of 


S 


the project and on board? S 


e Funding. Is there money to cover all of the above? You don’t want funding to run out 
halfway through a project. 


BUILD A RELATIONSHIP 
Building a good collaborative relationship at the beginning will smooth the way for better 
research and better services. Researchers and CBO staff should hold an initial meeting and 
make an effort to understand each other’s skills, commitments, expertise and culture. Did 
the CBO program manager major in biochemistry and then decide he wanted to work 
directly with people and not in a lab? Did the researcher start off in a neighborhood health 
clinic and then go back to school to learn about research? How did each participant come 
to their current work in HIV? 

Develop trust by spending time together, sharing plans and aspirations and listening openly 
to alternative points of view. Establishing a foundation of trust is one of the most important 
_ aspects of successful collaborations. Trust can help overcome many barriers. When trust is 
present in a relationship, communication is clearer, relationships flourish and partners are 
more open to input from the other. Without trust, communication is delayed or reluctant, 
expertise and credibility are doubted and both partners resist learning and growing. 

Researchers should visit the CBO to see programs in action and to develop a level of 
comfort with the culture of the community. Researchers should also note what resources the 
agency has such as computers, software, e-mail and a fax machine. If the research project will 
take place off-site, the researcher should visit the intervention site as well. Conversely, the CBO 
staff should visit the research offices. Researchers might have resources that could be made 
available to CBO staff, such as a university library, free access to the Internet or copy machines. 


MAINTAINING A COLLABORATION 
All relationships, especially cross-cultural ones, hit bumps along the way. Even CBO staff 
and researchers who are experienced collaborators and have followed all the best practices 


outlined so far in this manual will run up against problems. Here are some ideas to prevent 
and overcome these common barriers. 
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Regular Meetings 

It is vital to set up regular meetings throughout the collaboration. 

These meetings are important not only to track the progress of = 
the research, but to build trust, get to know each other and 
maintain relationships. 4 


PLEL OC NOT £4 


¢ Set a specified time — e.g., Monday afternoons at 2:00 — A researcher and CBO targeting 


and try not to change it unless it is absolutely necessary. underserved communities of 


color collaborated to look at 


e Researchers should meet at the CBO in order to get a 


sense of the organization and the programs. family histories of injection drug 


users. Scheduling meetings with 
research and CBO staff was difficult 


due to the hectic lives and competing 


e Having food at meetings is a good idea, whether people 
bring their lunches or each takes turns bringing bagels 
or donuts. 


interests of everyone involved. Also, 
e Depending on the topic of each meeting, there should 


be someone designated to take notes. At the end of 
each meeting, this person should go over who does 
what and the timelines so that everyone agrees with 
their tasks. 


outreach workers had difficulty 
concentrating on the research project 
when meeting because they were 
thinking about their other obligations. 
They decided that the best time to 


get everyone together was early in the 


e Make sure to schedule enough time so that 


everyone who needs to has time to speak and 


get their questions answered. morning, so the researcher went to the 


agency at 8 a.m. every Wednesday, where 
° Meetings should occur frequently (once a week) 


at the beginning of the collaboration and other 
times of intense work (such as data analysis), 
but can slow down when needed. 


she met with the project director and 
outreach workers. Each meeting began with 
a personal check-in, where people could talk 


; about what was going on in their lives. This 
e Don’t make assumptions about how much 


each person knows and understands. Even if a 
researcher has seen the intervention and trained 
interviewers, he may still need reminding about 
the details of the intervention. 


helped diffuse tension, build trust and allowed 
everyone to focus on the task at hand during 
the rest of the meeting. 


Involving the researcher, CBO staff and 


Throughout the collaboration it is important people in all stages of drug recovery ensured the 


to repeat and possibly redefine roles and respon- 
sibilities. For example, both partners might 
have decided at the beginning of the project 
that the researcher or someone on her staff 
would enter the data once collected. But what 
‘f one of the outreach workers who administered 
the questionnaire got excited by the process and wanted to join in the data entry? Allowing 
roles to expand and change maximizes the learning experience of everyone involved. 


collaboration benefited from lots of expertise in 
lots of different areas such as substance abuse, 


community and research. 


Training 
As both researchers and CBO staff are entering a domain that may be new to them, orienta- 
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tion and training are key. While it is fairly obvious that this should occur at the beginning of 
the collaboration, additional training may be needed throughout the relationship. Specialized 
training may be called for later in the collaboration on topics such as interviewing techniques, 
confidentiality or data analysis. Often, this is carried out by the researcher; however, the 
trainer should be familiar with the CBO environment and respectful of the different 
strengths and experiences between CBO staff and researchers. 


Researchers should give CBO staff an introduction to theory and practice of research and 
evaluation. Be sure to set aside time for questions and small group interactions. CBO staff 
should give researchers an introduction to program delivery issues such as funding cycles for 
services, Community Planning and its impact on programs, population needs and nuances, 
service delivery practices, etc. , 

Training will also be needed as staff turnover occurs and new staff are hired. Both 
researcher and CBO staff need to commit to training new staff as they arrive to ensure that 
the research skills of the agency do not leave with the employee. Having support from the 
CBO and, if possible, from a facilitator or other research teams in a consortium, can help 
smooth the transition between staff. It is a good idea to have a booster training midway 
through the collaborative project. If disagreements or misunderstandings occur, if research 
results aren’t matching what the CBO staff or researcher was hoping for, or if the research 
team is stymied by any data collected, it might be a good time to go over what has been 
learned and make sure everyone is looking at the situation from the same knowledge base. 


Support 
Adequate funding and support in all levels of the CBO and the research institution are 
essential in maintaining an effective collaboration. 


Monetary support. Research is routinely conducted at academic research centers — 
that’s what researchers are paid to do. But many CBOs do not routinely conduct research, 
and most CBO staff are not paid to do so. Therefore it is important to budget appropriately. 


¢ All CBO staff involved in the collaboration should be paid for their time, 
including time attending meetings and trainings. G 


¢ For CBOs, budgets should include funds for the intervention (if one 
is being designed or tested), the evaluation and for dissemination of 
study findings. 


¢ For researchers, budgets should include meeting times, training 
and dissemination. 


¢ Both should recognize that collaborations are time-intensive and may take longer or 
require more personnel time than working alone. 


¢ Both should commit to multi-year funding to ensure a smooth process and continuity. 


¢ If at all possible, include funders in the collaboration process. Invite funders to 
collaboration meetings and provide regular feedback on the progress. 


CBO support. Again, research is not a part of most CBOs’ scope of work. In order to 
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ensure support during the collaboration and to build capacity for research in the CBO 
after the collaboration, a wide array of CBO staff must be involved. The agency as a whole 
should be receptive to and aware of the value of research. Ensuring buy-in from various 
parts of the agency will also help when staff turnover occurs. Encourage buy-in by: 


i 
e Inviting Executive Directors, department heads, front line workers and administrative 
staff to meetings from time to time. 


e Reporting back to the rest of the agency at timely intervals on the progress 
of the collaborative research. 


¢ Including the agency in dissemination plans. 
¢ Focusing on how the research can help the agency learn about its clients or programs. 


Collaboration support. At some point all relationships hit rocky areas, and collaborations 
are no exception. When collaborations get stuck, it is recommended that a neutral party be 
brought in to offer support and facilitation. For researchers and CBO staff working without 
a facilitator or as part of consortia, this is especially important. In the facilitated or consortia 
models of collaboration, a person is already designated to support the relationship. This 
person can help clarify roles and responsibilities, open lines of communication and mediate 
any disputes. 

Working in the consortia model of collaboration also provides support to CBOs and 
researchers by allowing for peer support. Meeting with others involved in collaborations 
gives CBO staff or researchers a chance to compare their processes and brainstorm solutions. 
Just as peer support and peer education are powerful educational tools in promoting healthy 
behaviors, peer support empowers members of the collaboration consortia to learn new skills. 


Flexibility 

Chaos happens. Working in a collaboration does not offer protection from chaos, 

but rather doubles the chances of encountering it. Researchers and CBO staff need to be 
especially flexible in order to survive the bumps along the road. Flexibility is needed not 
only in interactions within the collaboration, but within the research project itself. 
Research questions may need to be altered, surveys rewritten or data analysis re-reviewed 
to accommodate real life. Researchers and CBO staff need to keep an open mind: 


e Ensure that all participants are committed to the services being offered, to the 
evaluation project and to the consortium. 


¢ Understand that CBO staff and researchers have different timelines. They also have 
different problem-solving techniques. 


© Collaboration can be a learning experience for all: CBOs learn about the process and 
skills of evaluation and researchers learn about the needs of clients in a service agency. 


e Be prepared to compromise and try new approaches. Questionnaires may end up 
asking different questions than a researcher expects; interventions may have different 


components than a CBO project director expects. 


e Keep your mind open to alternative viewpoints and solutions. 
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© Build a back-up research question into the research design. 


¢ Keep a close watch on the progress of the collaborative research and make sure 


timelines are set and adhered to. 


It’s important to keep reminding yourself that the reason for engaging in a collaboration 
is not to make more work for yourself (although it may sometimes seem that way). The 
reason to engage in collaborative research is to improve the services and research we conduct 
so that together we are more likely to help slow the HIV epidemic. Following these best 
practices can help the collaborative research project flow smoothly and ensure that each 
partner has a chance to learn and to improve their work. 

The next chapters describe the nuts and bolts of conducting research collaboratively: 
defining the scope of work, collecting data and conducting data‘analysis. 


KEY POINTS: STARTING A COLLABORATION 


In order to get the best results, these conditions should be in place for successful 
research collaboration: 


Goal. The goal of collaborative research is not simply to collaborate, but to conduct better 
research to improve services to help slow the spread of HIV. This common goal can help 
keep collaborations on track. 


Commitment. There needs to be a strong desire within your organization to collaborate and 
conduct research. Ideally, there is buy-in from all levels of staff. 


Desire.to Learn. Each partner should be interested in learning from the other and the 
process of collaboration. Each should acknowledge the expertise and experience of the 
other and be willing to admit that there are things they do not know. 


Time. Collaboration is labor-intensive and everyone involved needs to be prepared to devote 
the time necessary to make collaboration work. There is no quick and easy way to collaborate. 


Support. Each organization needs to have the capacity to take on a collaborative research 
project and provide the support it will need in terms of training, meeting and administrative 
and technical support. 


Funding. Collaborating will likely require additional funds for each organization. Make sure 
that time and effort are written into the grant or covered by current salaries. 


Flexibility. Research and CBOs have their own bureaucracies and are affected by a myriad 
of outside influences such as policy changes or natural disasters. Partners need to expect 
challenges and be willing to apply resources to work them through. 


Adapted from: Beyond Our Borders: A Guide to Twinning for HIV/AIDS Organizations. 
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CASE STUDY 


AGAINST ALL ODDS: CONDUCTING RESEARCH IN A CBO 


7 


by Drew Feraios 


s the deputy director of a grassroots CBO, | began a collab- 
oration with CAPS researchers not so much for evaluation 


expertise, but more as part of our ongoing struggle to fund 


our programs. 


During the summer of 1996, I was working at 
AIDS Project East Bay (APEB), an agency 
that provides HIV prevention and services 
programs in one of the more underserved 
communities found in the Bay Area. We 
were still reeling from the election of the 
Republican majority and the subsequent 
passage of Welfare Reform. So when we 
received an RFP from Northern California 
Grantmakers (NCG) for an evaluation proj- 
ect that would pair our agency with a CAPS 
scientist, we naturally applied for it to help 
ensure the agency’s survival. 


We had just lost funding for our prevention 
program targeting substance users, so this 
new REP provided us with the means to 
retain an exceptional employee and a needed 
program. What the program was mattered far 
less than the fact that a new source of funding 
might be in the offing. 


The first culture clash between our agency 
and the CAPS researchers came when we 
tried to describe the program and how we 
wanted to evaluate it. After several meetings 
in which we tried to finalize a research ques- 
tion and a methodology, we came to the 
consensus that our program was something 


Case Study: Against All Odds: Conducting Research in a CBO 


we pieced together as best we could from as 
many funding sources as we could find to do 
something we knew had to be done: provide 
prevention services to substance users in 
Oakland. Once we found the language to 
explain how the program worked, CAPS 
scientists helped us develop an evaluation 
methodology in harmony with the grassroots 
nature of the program. 


The second culture clash had to do with the 
fact that CAPS staff relied upon e-mail, 
voicemail and a computer network. We at 
APEB were severely limited by our lack of 
‘90s technologies. Even when we tried to 
work from computer disks, our version of 
Word could not read documents from 
CAPS! We found that the best way to keep 
the channels of communication open was to 
meet once a week at 8:00 AM; these meetings 
continued long after funding for the project 
dried up. 


That we succeeded at all in completing the 
evaluation is a testament to those of us that 
made up our team: Diane Binson and Bonnie 
Faigeles at CAPS; myself, Oliver Saunders, 
Marvin Williams, and Lynn Mayberry from 
APEB. More than a year after the funding 
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ended, we still meet and are close to develop- 
ing manuscripts from rich qualitative data 
about the lives of substance users in Oakland. 


For me, the result of our collaboration was 
twofold: 1) We finally had data about the 
magnitude of the social problems faced by 
our clients to share with funders and the 
public, and 2) I realized I could not continue 
working in services, and that research 
offered me a different path to address the 
same issues. At the final meeting of the 
CAPS/NCG Collaborative Research Con- 
sortium, | presented the data from our project 
by painting a picture of the lives of those we 
had studied. The room was silent and 
engaged as I described sexual and physical 
abuse and violence, parental alcoholism, 
prostitution, bouts of homelessness, along 
with stories of hope from substance abusers 
that had forged new healthy lives for them- 
selves. At the very end, a woman in the 
audience revealed herself to be one of the 
subjects in the study, and with tears in her 
eyes, she told me that our team had “told it 
like it is.” Nothing in all my years at APEB 
had so moved me. After the project was over, 
I was hired by the Center for Health Improve- 
ment and Prevention Studies (CHIPS) to be 
the project director of a federally funded 
study. Once I got to see just how much of the 
work we do in grassroots nonprofits is simply 
to make up for our lack of resources, I just 
couldn’t do it anymore. 
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Funders often prevent CBOs from developing 
the infrastructure necessary to increase their 
productivity by offering progressively smaller 
funding units and progressively more intense 
restrictions on administration. This also 
increases the burnout rate among CBO 
employees, which reduces an agency’s ability 
to serve its clients with their myriad 
concerns. The CAPS/NCG collaboration 
and my work at UCSF provided me with 
the hope of illustrating these community 
concerns and offering ways to ameliorate 
them. Today we at CHIPS are preparing to 
hit the field with an innovative study look- 
ing at smoking, drinking, HIV and drug risk 
behaviors among primary care patients in 
the Bay Area. After more than a year of 
working at UCSE I continue to find hope 
that research may guide policy decisions that 
help bring about social change. 


Case Study: Against All Odds: Conducting Research in a CBO 


Chapter 


Developing 


a Research Question 


The key criterion of a good research question is: do you care about the answer? 
Do you believe that the answers to the research question will help ‘you think about your 


work in a new way or provide a critical piece of information? 


Chapter 


ne of the first tasks in deciding the goal of the collaborative research project is 

determining the research question: what do we want to know about the population 

we work with, or the services we provide? The key criterion of a good research 
question is: do you care about the answer? Do you believe that the answers to the research 
question will help you think about your work in a new way or provide a critical piece of 
information? Each collaborator brings his own expertise, and all of it will be useful in 
defining the research question. Generally, the CBO brings information about the program 
or services being studied and the daily realities of the target population, and the researcher 
brings expertise in evaluation and research procedures. Of course, some collaborators have 
overlapping experiences. Some CBO staff have been doing research for a long time, and 
some researchers are as familiar as the providers with a local target population. Being open 
to all the expertise among the collaborators will serve the goal of defining an answerable 
research question and developing good procedures for data collection. 

While identifying a research question, it is important to determine whether or not it can 

be answered. Many times an initial research question cannot be answered because: 


e There is not enough time or resources to carry out the research needed to address the 
research question. 


e The question is more complicated than initially thought. 


e It is not clear how to collect information that would answer 
the question. 


e The question is not specific enough. 


In these cases, the research question has to be restated or rede- 
fined. For example, it is reasonable that a CBO would want to know if 
their prevention program works — this is “outcome” research. But most 
programs can’t assess a sufficient number of clients in a short enough period of 
time to be able to answer the question, “Does our program reduce HIV infections?” 
Nevertheless, there are plenty of other research questions about the program that are worth 
pursuing that can be answered. For example, instead of asking about outcome, it might be 
good to start out by asking if the program addresses the needs of the population it’s meant to 
serve, or if staff are implementing the program as it was intended to be done — these would 
be examples of “process” research. 

Collaborators should be flexible and prepared for changes to the research question. 
Having a backup question, or a secondary research question, is also valuable, since sometimes 
the primary question cannot be answered definitively. 
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FORMULATING A RESEARCH QUESTION | | 
Here is a step-by-step example of how one CBO and researcher team came up with a specific 
and manageable research question that got to the heart of what they wanted to know. 


The program manager of a CBO providing drug abuse 
treatment was concerned that her agency’s clients weren’t 
getting HIV antibody tested, despite the fact that they were 
at considerable risk of HIV infection and that if they found 
out that they were HIV+, they could access early interven- 
tion services. The first question was: 


oO 


FELD MOT es 


Why don’t our clients get HIV tested so those who are HIV+ A CBO that puts on plays to 


can start early intervention treatments for HIV? educate adolescents in schoo! 


about HIV wanted to compare the 


At the first project meeting, the program manager, 
researcher and several case managers discussed the 
research question. The researcher asked about the 
CBO’s clients and wondered if they really wanted to 
know about all the clients or a particular subset. The 
case managers agreed that there were a number of 
reasons why certain clients had been unable or 
unwilling to be tested. One of the case managers 


impact of their play when teens and 


parents together saw the play to 
when teens alone saw the play. The 
CBO staff and researcher collaborat- 
ed to design the survey and implement 
the two interventions. However, few of 


noted that barriers were different among persons who the parents showed up for the interven- 
believed they might be HIV+ as compared with those 
who believed they were HIV-, and that drug users 
who believed they were HIV+ were more willing to 
reach out for help. They all agreed to focus on 
clients who believed they might be HIV+. The 


research question then changed to: 


tion with teens and parents together. 
The researcher and CBO staff 
discussed the problem with other collabo- 
rative research teams and discovered that 
almost all projects that involved parents 


had difficulty recruiting them into the 
Among clients who believe they are HIV+, what 
are the barriers to getting tested and to subse- 


quently obtaining early intervention treatment? 


study. Together, the researcher and CBO 
program director strategized ways to 


address the issue of parent/teen communica- 
The researcher noted that it might not be 
easy to differentiate between clients who think 
they are HIV+ and those who don’t, which 
would make recruitment difficult. He suggested 
that they try to recruit all injection drug 
user clients who had not tested for HIV, 
and that in the analysis they could look 
for any differences between those who 
thought they were HIV+ vs. those who 
thought they were HIV-. A case manager 


tion without involving the parents. 
The research team changed the research 


question to finding out the impact of the play on 


teens’ willingness and experience talking with 
their parents about sex and HIV risk. The project 
collected rich data on what teens want and get 


from their parents around sex and HIV risk. 


Page 28 Chapter 4: Developing a Research Question 


said that in his experience, barriers to HIV testing were very different for men than for 
women, that women who inject heroin were less likely to seek out services of any kind, 
and that more women seemed to be seroconverting than men. The program manager 
added that their clients were mostly male and she wanted to find out more about their 
female clients, even if they were harder to access. A case manager added that using 
incentives and offering child care might encourage women to participate. The researcher 
suggested they over-sample for women, that is, recruit equal numbers of women as men 
even though most of the clients were men. 

One of the case managers brought up the topic of early intervention, and how the 
project would address that. The researcher noted that previous research had identified a 
relationship between HIV testing behavior and belief in/access to early intervention. The 
case manager concurred that in his experience, clients who knew about early interven- 
tion or used medical services were more likely to test. Therefore, the group decided to 
include questions about belief in early intervention and access to medical care in the 
questionnaire. The researcher suggested that instead of making that the research ques- 
tion, they should have a broad-based question and then look for differences or patterns in 
the analysis phase. The research question was finalized as: 


What factors are associated with in-treatment injection drug users’ decisions not to test for HIV? 


Note that the conversation about the research question had implications for sample 
size and selection, data analysis, survey design, survey questions (topic areas) as well as 
the actual research question. Deciding upon a research question often overlaps the steps 
that follow such as deciding on recruitment strategies and designing survey instruments 
or interview guides. The next chapter will outline how to collaboratively begin the process 
of data collection and move into data analysis to get answers to the research question that 
will help strengthen programs and services. 


KEY POINTS: DEVELOPING A RESEARCH QUESTION 


© The mark of a good research question is that you care about the findings. 


e Have a backup or secondary research question built into the research project 
from the beginning. 


© Take time to discuss and hone the research ideas into the right research question. 


© Make sure that both the researcher and CBO staff are committed to the research question. 
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Chapter 


Collaborative Data 
Collection and Analysis 


Working together through the various steps to develop and implement 


data collection and analysis procedures will result in better procedures that deliver 


more accurate data. 


Chapter 


” 


he last chapter showed how defining the research question in a collaborative 

manner can also help guide the data collection and analysis phases of the research 

project. Research progresses in phases: developing the research question, designing 
the surveys or other instruments, collecting data, cleaning data and entering data, then 
analyzing and interpreting the findings. Because all these phases of research are connected 
and overlapping, collaboration is necessary at each stage to ensure the best outcome and the 
ability to collaborate in future stages. 


WHAT ARE DATA? 

Data are all the answers gathered during the research process. Most commonly data are 

in the form of text or numbers, such as answers to interview questions, scales from 1-5, 
numbers of partners in the last 6 weeks, etc. There are also different levels of data you can 
collect, such as data on individuals, groups, organizations and communities. Data can be 
qualitative, that is, in narrative or text form, for example, information collected from focus 
eroups or in-depth interviews. Data can also be quantitative, that is, numerical, in the form 
of numbers, for example, survey data and data from epidemiological reports. 


HOW ARE DATA COLLECTED? 
Data can be collected through surveys, interviews, observations, focus groups, forms, 
reports and materials that were constructed for other purposes, such as client intake forms. 
The research question will dictate the kinds of data needed and the ways in which it is to 
be collected. Deciding how to do it depends on the research question and how much is 
already known about the population. Procedures for data collection must be developed in 
detail and require the commitment of all the collaborators. Among the details to be 
defined are who will do what and how the procedures will be carried out from 
collecting the data to having it ready for analysis. Examples of tasks to be 
defined are: deciding what kind of data is most appropriate for answer- 
ing the research question, constructing the interview or survey ques- 
tions, typing and copying the survey, recruiting participants, getting 
food or incentives for focus groups, conducting interviews, entering 
data into the computer, reading through transcripts, transcribing 
tapes, etc. 
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WHAT IS THE PROCESS FOR COLLECTING AND ANALYZING DATA? 

Data collection and analysis begins with conceiving the research question. The type of 
analysis conducted is linked to the type of data collected, which in turn is determined by 
the research question. The collected data may be qualitative, quantitative or a combination 
of both. The next section is a guide to the different phases of the research. (Please see 
Appendix D for a glossary of research terms.) The example of the research project studying 
drug users who do not test for HIV (described in the previous chapter, pg. 28) will be used 


to illustrate each step. 


Data Collection Instruments 

Once the research question is decided, the next step is to decide how to collect the informa- 
tion needed to answer that question. This can include writing detailed questionnaires, 
outlining focus group questions, designing interview guides or searching for existing archival 
data. Generally, the researcher provides guidance on design of instruments to make them 
compatible for analysis, and the CBO staff provides guidance on content and wording of 
questions. If a project decides to use questionnaires for gathering quantitative data, this 
phase of the research may take quite a while, as making sure the correct questions are asked 
is key. If the project will be collecting qualitative data, this phase may be shorter, with the 
bulk of the work coming at the data analysis phase of the project. 


In our example, the research team wanted in-depth interviews with clients who had not tested 
for HIV. Because they wanted to do comparisons between men and women, and those who 
believed they were HIV+ and HIV-, they wanted to design a quantitative survey. At the end of 
the survey, they wanted to include more open-ended interview questions to allow clients to talk 
about their experience with HIV and issues in their lives that affect health seeking behaviors. 
The team spent a long time designing the questions for the quantitative survey, as CBO case 
managers wanted to make sure a wide variety of questions were asked in language that their 
clients would understand, such as history of violence and abuse, work experience, housing 
status, relationship with partner and drug use with partners. 


Once the instruments are designed, they should be pilot tested. This may entail holding 
a focus group of potential clients to discuss the questions and issues addressed. It may also 
entail conducting a few sample interviews (sometimes with the outreach workers who will 
be administering the survey) to make sure the questions are understandable, elicit usable 
answers and use culturally and educationally appropriate language. Any feedback can then 
be incorporated into the survey before beginning the data collection phase. 


Recruit Participants and Collect Data 

The next step is recruiting participants and conducting interviews, focus groups or other data 
collection. During or before this step, several trainings can be set up, including how to 
administer a survey, protect client confidentiality, store data, and conduct basic data analysis. 
The collaborative research study should rely on the strengths of all research partners, and 
those team members who wish to learn new skills should have access to appropriate training. 
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In our example, the research team suspected that it might be difficult to recruit female clients for 

the study. They also knew that the interview would be long and detailed, and require that the 

clients trust the interviewer. They decided to allow extra time for recruitment, and to allot more 

funds for providing incentives that would attract clients. One of the case manager’s daughters was 
° an experienced babysitter, so they provided child 


care for any client with children. The researcher 
FIELD NOTES 


knew that other projects at the university had a 

surplus of incentives, and was able to get phone 
cards to give to participants and cookies to serve 
during the interview. The outreach workers were 
eager to conduct the interviews to learn more about 
their clients as well as to earn extra money. The 
researcher set up a training on client confidentiality, 
interview skills and data cleaning for all interviewers. 


A collaboration between a researcher and an 


agency targeting underserved communities of 


color wanted to design a survey that accurately 


reflected the experiences of the community. 


Often, as is the case in the example above, 
CBO staff collect data through conducting inter- 
views with clients. However, this changes the role of 
CBO staff and the dynamic between them and clients. 
Instead of being a sympathetic ear, they are now asking 
questions and maintaining appropriate distance from 


However, designing the survey took longer 


than the researcher expected, and there was 


disagreement between the researcher and 


CBO staff on wording. 


For example, the researcher developed 


clients so as not to influence responses. Alternatively, 
staff from the research side of the collaboration may 
conduct data collection efforts, requiring that strangers 
come into the service arena asking questions of clients, 
which also might upset the balance and cause distrust. 


questions about being hit by family 


members hard enough to leave a mark or 


wound. The original question was: “Did 


you consider that physical abuse? Yes or 
No.” Some of the CBO staff thought it 
would be better to ask whether the 


If clients are getting paid to answer a survey, for exam- 


ple, that can have an impact on the relationship between 
client and case manager. If a researcher is supervising CBO 
staff in data collection, it is important to understand roles 
and negotiate supervision methods. CBO staff may have very 
different styles of communication and researchers need to 
follow the appropriate channels of supervision that are built 
‘nto the CBO. Taking advantage of the different perspectives 
and experiences of team members in constructing how the 
data will be collected and determining who does what can 
lead to insights that would otherwise go unnoticed. 


respondent thought of this as “disci- 


pline.” After a long discussion with 


different opinions, the question was 


reworded to ask whether the respon- 


dent thought being hit was “ohysical 


abuse, discipline, or both.’ 


The revised question generat- 


ed interesting data that better 


Enter and Clean Data 
After the data have been collected, they are usually entered 
into a computer database. Data cleaning and quality control 
occur before, during and after data entry. Before data entry, ques- 
tionnaires should be cleaned, that is, reviewed for responses that 
don’t make sense or are out of range; for example, if someone lists 
their age as 150. For quality control during data entry, data should 


captured the realities of those 


being interviewed. The CBO 


staff’s expertise helped shape 


more accurate survey questions. 
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be entered into the computer twice, preferably by different people, and compared. This 
verification assures consistent data entry. After data entry, the file can be re-checked by a 
computer program for out of range and inconsistent responses. For qualitative data, the : 
equivalent of data entry is transcribing the text of a tape-recorded interview and/or writing 
up field notes. 

Data entry can be a time-consuming task if the research project has reached a large 
number of clients with a long survey. Data entry can be done by CBO or research staff, or it 
can be sent to a professional data entry organization or professional transcriber, to which the 
researcher should have access. If a project is considering conducting many interviews, it 
might be a good idea to budget for an outside organization to do data entry. 


In our example, the CBO had a summer intern whom they recruited to do the data entry. 
The program manager volunteered to do the data cleaning so that she could read through 
the interviews and get a sense of what clients were saying. The researcher also participated 
in some data cleaning and provided quality control of the data entry. 


Analyze Data 

For quantitative data, the first step is to look at frequencies, that is, the number and percent 
of people who answered each particular response, for each question or variable. Variables 
include demographics (e.g., age, race/ethnicity, gender, income, etc.), behaviors (e.g., 
condom use, accessing needle exchange) and attitudes (e.g., self-esteem, feelings about 
condoms). Frequencies provide descriptive information about the population, including who 
attends the programs or the prevalence of certain risk behaviors. After frequencies, certain 
variables can be chosen for comparison. This allows for analyzing the data by subgroups, 
such as comparing males and females, different age groups, etc. Comparisons can also be 
made across time, if longitudinal data is collected. More complicated comparisons can be 
conducted to determine the relationship between more than two variables. These types of 
analyses are generally done by the researcher or statistician because they require statistical 
“computer programs such as SAS and SPSS and specific technical expertise. It is important 
for CBO staff to have access to data and software at all times, especially once the research 
project has ended. This will help them if they request further analyses or hire their own data 
analyst later. 

For qualitative data, the first step is to read the transcripts and/or field notes. It’s a good 
idea to write down ideas that come to mind while reading the transcripts, including questions, 
surprises or any common themes that arise between transcripts. If there is a transcript that is 
radically different from others, it may be worthwhile to spend some time examining how it 
diverges from the rest of the data and what made it different. Because qualitative data 
involves words instead of numbers, the raw data in the form of transcripts is often more 
accessible and understandable than quantitative-data. However, it is necessary to develop a 
data analysis plan, a systematic way of handling the text. Some people who work with a lot 
of qualitative data prefer to use a computer program specifically designed for this type of 
data such as NUD-IST, ETHNOGRAPH or ATLAS-TI. All of the text can be put into the 
program, and then certain text and terms can be flagged or noted. 

Another way to manage qualitative data is to summarize each interview into a more 
manageable form, usually a couple of pages. Researchers and CBO staff can jointly decide 
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the areas they want to focus on and then summarize those areas of the transcript in 14 
pages. This way, they can have the data in a more manageable and workable form, instead 
of working with a 30+ page transcript. Summary sheets can then be used to draw conclusions 
and refer to the original transcript for key quotes. The process of reading and discussing the 
interview transcripts and/or field notes can allow for equal participation of all parties. In 
best practice, both CBO staff and researchers will provide unique perspectives on making 
sense of the interviews. 


In our example, the research team collected both quantitative and qualitative data. Once the 
quantitative data were entered into the database, the researcher took over, giving it to her 
statistician to analyze the data and run frequencies. The next collaborative step was to interpret 
the data. Both researcher and program manager felt good about the data they collected, and 
had some sense of what they might find out from reading the surveys and doing data cleaning. 


For the qualitative data, the research team divided up the transcripts from the interviews among 
the research and CBO staff and each member was responsible for summarizing the interviews 
into 2-4 pages. There were six members of the team (researcher and statistician, CBO program 
manager and three case managers) and each member was given three interviews to summarize. 
The summaries were all verified by the researcher. Halfway through the process, the CBO’s 
summer intern expressed interest in learning to summarize, and one of the case managers left the 
agency, so the intern stepped in to the analysis team. The team then met and jointly read 
through each summary. They created a sheet that listed different types of barriers to HIV testing 
that clients mentioned, such as childcare, shame, fear, etc., and then summarized the areas 
where the participant discussed each of these issues. They also started another list of issues that 
popped up sporadically but were interesting, such as drug users injecting their partners. 


Interpret Data 

Interpretation gives meaning to the data analysis and allows for drawing conclusions about 
the participants and/or the program. This is the part of data analysis that incorporates other 
research and experience into understanding what the data mean. The input of both 
researchers and community service providers is especially important in this phase as they 
have different experiences and knowledge that can make the findings clearer. In the inter- 
pretation phase, the research team can work with the statistician to look at different aspects 
of the data, or reinterpret the data using other variables. 


In our example, the research team got rich information about their clients and barriers to 

HIV testing. Through their quantitative data they learned that most clients knew about early 
intervention and treatment for HIV, and that had no effect on their desire for testing. One case 
manager suggested they look at whether there was a difference between men and women, or 
between those who had access to health care and those who didn’t. Again, there was no effect. 
This surprised the program manager and made her take a closer look at other barriers. The 
qualitative data gave lots of information about clients’ lives and barriers to HIV testing as well. 
One surprising finding was that women who hadn’t been tested were very eager to get tested, 
whereas men preferred not to test. The reasons women gave were more altruistic, like caring 
for their family and taking responsibility for their own and their partner's health. 
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BARRIERS TO COLLABORATION 
Research collaborations have the potential to use the best 
skills, experiences, and ideas of all participants. When it 
comes to data collection and data analysis, however, there 
are many myths that can become barriers to having everyone 
participate. One myth is that data collection is something 
that only researchers can do. CBO participants may think 
that they don’t have the expertise, and full responsibility for 
the development and implementation of data collection 
procedures can fall to the researcher. However, CBO staff 
are also experienced in data collection. Though they may 
not call it data collection, CBO staff collect all kinds of 
data on a regular basis in the form of client intake forms, 
reports, site observations, evaluation forms and 

outreach notes. These data help the CBO make deci- 
sions, develop new programs and generate funding to 
support services. Also, because the CBO is intimately 
connected to its population, the staff are typically in an 
excellent position to provide important information 
about how to set up and implement the various stages 
of data collection (e.g., suggesting appropriate word- 
ing for constructing questionnaires), or choosing 
recruitment venues and procedures. 

Another barrier to collaboration is that it 
takes longer to design surveys, collect data and 
analyze data collaboratively than when it is solely 
researcher-conducted or CBO-conducted. More 
time is necessary for training and explanation 
and working through cross-cultural differences. 
Each member of the collaborative research project 
must be committed to the benefits of collabora- 
tion, and allow the extra time it requires. 

Working together through the various steps 
to develop and implement data collection and 
analysis will result in better procedures that 
deliver more accurate data. Recognizing and 
tackling the myths each group may have 
about the other and their own roles will go 
a long way in improving the quality of the 
procedures developed. 
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LEADERSHIP 

Even though the process of data collection and analysis may be collaborative, it is important 
to have clear leadership. Clear leadership enables the process to move forward and facilitates 
timely decision making. The leader may be either the community service provider or the 
researcher. If the researcher is the leader, they should ensure CBO participation by providing 
some basic training about research and analysis and drawing out CBO staff’s opinions and 
ideas. If the CBO is leading the effort, CBO involvement may be more organic than if the 
researcher were the leader. However, training and encouraging CBO staff participation is 


still useful. 


FUNDING 

Money can be the driving or limiting force behind a collaboration, especially in the data 
collection and analysis phase. Collaborative data analysis certainly requires more time than 
the alternative. Staff need to be compensated for their work in research to allow adequate 
time for their participation. Researchers need to be funded to train CBO staff to enable 
their participation in the process. The party that brings the money to the table often has 
the most say in the process; literally, how everyone will spend their time. Because CBO staff 
usually know less about data analysis than researchers, they may initially have less input 
into the process and feel that their voices are not as important. If the CBO staff also brings 
funding to the project, through their own funding or by seeking additional outside funds, 
they may feel more entitled to express their opinions and have input into the project. This 
can be a way of mediating the imbalance of research knowledge in collaborative data analysis 
and interpretation. 

Data collection and analysis gives the answers to the research questions, what you’ve 
been anticipating and working towards since you started planning this process. Because all 
of your research work has led up to these findings, they are very valuable. Jointly conducting 
data collection and analysis is crucial in making these findings as rich and useful as possible. 
It’s especially important that CBO voices inform the analysis process to connect the data to 
the community. In the next chapter, you can see how to take these findings from your data 
analysis and use them to impact your programs or research. 
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KEY POINTS: DATA COLLECTION AND ANALYSIS 


Data collection and analysis start with the design of a research question and don’t end until 
the findings are implemented and disseminated. Some of these lessons learned are similar to 
what has been presented in other chapters, and they bear repeating. Others are specific to 


this phase. 


e Both quantitative and qualitative data can be successfully collected and analyzed in a 
research collaboration. Quantitative data require more time,spent during the survey 
design phase, and qualitative data require more time in the analysis phase. 


© Clear leadership and clearly defined roles are very important. Each step of data collection 
requires a specific person to be responsible for carrying out the task, as well as oversight 
by a leader. 


e Everyone in the collaboration should be prepared through extensive training, especially 
regarding confidentiality, interviewing, and procedures for data collection. 


e Researcher and/or CBO staff turnover is almost a given, so it is best if plans are made for 
orientation and training of new staff. 


e Meeting regularly is required. Research and CBO staff must meet regularly during 
data analysis. 


e Allow for adequate time; collaborative data analysis is time intensive — include this in 
planning and budgeting 


e Decide together when to stop. Sometimes you must stop collecting data no matter how 
far short of your goal you are; other times, a minimum number of interviews may be 
required no matter how long it takes. 
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Cause STUDY 


WORKING WITH YOUTH IN HIV PREVENTION RESEARCH 


by Gary Harper 


have found that working on the formation and maintenance of 


community-university collaborations is one of the best ways for 
me to use my talents in the fight against AIDS. I was part of the 


collaborative research initiative 
at CAPS as a junior researcher 
paired with staff from Tri-City 
Health Center. 


Our research project looked at suburban street 
youth and their HIV prevention needs. It was 
a great experience for me as I saw firsthand 
the multiple benefits that could be derived 
from such collaborative efforts for myself, the 
agency staff and the youth themselves who 
were involved in the research project. 


When I moved from San Francisco to 
DePaul University in Chicago, I wanted to 
be able to connect with the HIV prevention 
community and recreate the positive experi- 
ence that I had at CAPS. In September 
1997, a graduate student and I developed a 
community-university partnership with 
Project VIDA, a nonprofit community-based 
HIV/AIDS service organization, in order to 
evaluate two of their HIV prevention 
programs for youth. The collaborative part- 
nership has now expanded to include more 
students and agency staff members, and we 
are now working together to evaluate six 
programs within the agency and to conduct 
trainings with Latino-focused community 
agencies throughout the Midwestern U.S. 


Case Study: Working with Youth in HIV Prevention Research 


Teen outreach workers at Tri-City Health Center. 


When the partnership began, it was clear that 
the Project VIDA team was extremely famil- 
iar with the target population and was able to 
provide us with a profile of the participants 
that were attending their programs, while we 
from the DePaul team were knowledgeable in 
conducting sound evaluation and research. 
Thus, we were able to educate and train staff 
members in how to develop and implement 
appropriate and useful evaluations. We all 
had a genuine interest in HIV/AIDS preven- 
tion among female adolescents and gay/bisex- 
ual/questioning young men (the two groups 
targeted in the initial efforts). This shared 
sense of direction and commitment helped us 
to focus on the ultimate outcome, which was 
to help protect these young people from 
becoming infected with HIV. Overall, 
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communication between both teams was 
essential, especially in the early stages of the 
collaborative partnership as we worked 
together and made important decisions as a 
team so that all members felt they were 
having significant input into the process. 


Participation in this collaboration has 
allowed me to give my university students the 
opportunity to learn about HIV prevention 
and evaluation by actually being involved in 
“real world” experiences. This was my first 
university appointment and I did not want to 
repeat the errors of many professors by talk- 
ing abstractly about social issues such as HIV 
and never connecting the information to 
“hands-on” experiences. Both graduate and 
undergraduate students have been able to 
learn firsthand about the realities and chal- 
lenges of conducting community-based HIV 
prevention work, as well as how to run an 
effective process evaluation that assists in the 
improvement of community-based programs. 
They have gained great insight into how real 
life prevention works in a community setting 
and about the multiple issues that must be 
addressed when doing such work. 
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Working with Project VIDA also has helped 
me to have a more realistic understanding of 
how cultural and environmental factors have 
a tremendous influence on young people’s 
ability to protect themselves from HIV infec- 
tion and other negative health outcomes. | 
have seen the devastating impact that the 
lack of culturally appropriate HIV preven- 
tion services has had on this community, and 
recognize the urgent need for the develop- 
ment and expansion of such programs. My 
participation in the collaborative team also 
has given me the support to organize and get 
involved in social action activities in an 
attempt to address these social issues. 
Working on this initiative has reaffirmed my 
belief in the need for more collaborative 
relationships between CBOs and evalua- 
tors/researchers if we are going to move the 
science and technology of prevention and 
health promotion ahead to the point where 
we can positively impact greater numbers 
of people. 


Case Study: Working with Youth in HIV Prevention Research 


Chapter 6 


Incorporating Findings 


into Research and Service 


The only way for a collaborative research project to fail 


is to pay no attention to the research findings. 


Chapter 


he previous chapters have outlined the nuts and bolts of conducting the collabora- 

tive research project, from choosing a research partner to analyzing and interpret- 

ing data. Often, once data analysis is completed, funding ends and the research 
project is considered finished. But findings can’t make a difference unless they are applied to 
service provision and research practices. A key part of a collaborative research project is incor- 
porating and disseminating the findings. You've got the answers to what you wanted to 
know, now what are you going to do with them? 

Let’s say the research collaboration has lasted two years. Together, the research team 
wrote proposals, trained staff and volunteers, trained each other, strategized and finally 
succeeded in getting participants into the intervention. The team has even cleaned and 
analyzed data together. So far so good. 

But Reckoning Day has arrived, as it always does. The researcher and CBO staff both 
have so much invested in the project’s outcome that they're very nervous when they finally 
interpret the data. Both are wondering if they’re going to have to give up their jobs or if 
their communities are going to be left with no way to stem the tide of new HIV infections. 
What do the clients need? Does the program have any effect? Has the research project 
answered what they wanted to know? 


RECKONING DAY FOR THE CBO 

The bottom line is that the research results might be difficult to hear. The hardest part of doing 
research can sometimes be knowing what to do with the findings. The data might suggest a 
radical shift in programs or understanding of the target population. Individuals resist change, 
and CBO program staff are no exception. The results may suggest that a program was a 
smash success and that unsafe sex might be successfully reduced in your community bya 
large percentage. The results might say that the target audience needs exactly the kind of 
program the agency had hoped to implement. Or the data may say that a program failed to 
have any impact on the rate of unsafe sex, or that people want something completely differ- 
ent than what the agency has to offer. However, results are rarely that straightforward. 

The good news is: you can’t really lose. Knowing that the program had a modest impact 
is still valuable information. If the findings showed that the program isn’t as effective as 
hoped, there are still two options which may serve the community well: changing the 
program design, or even shutting down the program. If the findings showed that the target 
audience needs a different kind of intervention, another option is to start a new program. 


Starting Programs 
In the best of circumstances, programs start Up because there is data to demonstrate a need for 


them. The data might be about the clients’ lives; or they might be from a program evaluation 
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that demonstrates the need for a new intervention. The data might be from qualitative 
interviews with a number of people who all convey the same concerns. 

It may be the case that after working on a collaborative research project to assess client 
needs, CBO staff and researchers decide to continue the collaboration to jointly develop a 
new program. This is an excellent way to translate the research findings into services. 
Researchers in HIV prevention and other areas of public health, such as injury control or 
anti-smoking efforts, can contribute their expertise and tell you what the scientific literature 
has shown works to change behavior — and how much change is reasonable to expect. 
CBO staff can bring knowledge of what has worked on the ground with their clients, as well 
as what other agencies in the community are doing. This can help ensure the team that it is 
not redesigning a program that already exists, or one that has been shown to be ineffective. 
There might be an existing effective intervention model that can be adapted to a particular 
target population. 

A word of caution: simply because the research findings indicate a need does not mean 
that the agency is the right one to meet that need. For example, the research might show 
(and it often does) that unless people have basic housing and food, they won’t prioritize 
reducing their HIV risk. If the agency specializes in HIV prevention, they’re probably best 
off providing referrals to housing and food agencies rather than trying to provide those 
services themselves. 


Changing Programs 
Changing programs is one of the easiest ways to incorporate research into services, in part 
because it’s the least risky and takes the least amount of time. Often, it is possible to make 
changes in an intervention quickly and have immediate impact on your services. 
Sometimes research findings go against some preconceived notions or intuition. For 
example, data may show that many men of color do not want separate educational groups 
but prefer coming to groups with men of all colors. In this case, the agency may need to 
start offering men of color a choice of single-ethnicity or mixed groups. Or data may show 
_ that individuals from different populations take similar amounts of risk but get infected at 
different rates. The results can now help with re-targeting the services most needed and best 
designed to meet a population’s needs. 


Closing Programs 

One of the biggest anxieties about conducting research, especially outcome or evaluation 
research, is that the findings will show an ongoing program to be ineffective. This leads to 
fears of loss of funding, inability to get new grants, the entire CBO closing and everyone 
losing their jobs. 

Hopefully that won’t be the case. By the time preliminary data have shown that a 
program is not performing as expected, the CBO staff and researcher most likely were 
already aware of this, which may be why they formulated the research question in the first 
place. It takes a lot of courage and commitment to admit that services or theories aren’t 


working, but it provides an opportunity to redirect resources to something that will be 
more effective. 
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RECKONING DAY FOR THE RESEARCHER 
It is not solely up to the CBO to implement collaborative research findings into their agency. 
There are several ways researchers should use the research findings to affect their research 


= . 


FIELD NOTES 


practices and agenda. Below are ways that 
researchers can learn and improve their work 
as a result of working collaboratively. 


Study components of successful programs 
Positive and negative results may mean differ- 
ent things to researchers and CBOs. Both 
groups want to benefit affected communities 
and save lives. A CBO will hopefully take 
the well-analyzed data, whether the results are 

positive or negative, and make sound program- 


A multi-service CBO serving suburban street 


youth originally wanted to compare the 


effectiveness of a same sex versus mixed 


sex small group session intervention. The matic decisions based on it. It is also important 


for researchers to think carefully about both 
positive and negative results. If they’re positive, 
make sure to understand why this particular inter- 


results were conclusive: very few youth 
came to either session. 


Instead of seeing this as a failure, vention worked so well. If the project was a success, 


the CBO staff and researcher decided work with CBO partners to figure out which 
components of the intervention were the most 
successful and may merit expanding. Similarly, look 
carefully at other parts which were probably not 
necessary and could be eliminated. How can posi- 
tive results be translated to understand the likely 
impact of this intervention on the epidemic as a 
whole? If the intervention relies on a lot of volunteers, 
and community interest in HIV is waning, could this 
program realistically be replicated again? Could the 
program be replicated elsewhere, given the cost? 


to gather data on what exactly this 
population needed. They studied inter- 
views that had been conducted with 
the few youth who did show up, and 
conducted more qualitative inter- 
views with youth on the street. 
They found out that youth 


wanted easily accessible drop-in Incorporate new questions into future surveys 


services and not fixed small Interpretation of results often hinges on the validity of 
group sessions. As a result of the survey questionnaire. For example, the survey may 
have asked how many sexual partners individuals had, but 
not have a way in the end of finding out what kind of sex 
they had with different partners, or what they knew about 
their partners’ HIV status. Or the survey may have included 
a question about family history of drug use and initiation, 
which the CBO staff suggested and the researcher would never 
have thought of. Working in collaboration with CBO staff can 
help researchers increase their knowledge of population needs 
and refine or create new kinds of questions. Researchers need 


to be sure to use what they’ve learned when designing future 


their findings, the CBO opened 


a drop-in center for teens 


that provides HIV prevention 
materials as well as 
referrals to job training and 


housing, food and clothing. 


surveys with this population. 
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Disseminate findings faster 
By now it is obvious that researchers have very different timelines than CBOs. Like CBOs 


who want multi-year funding because it assures their intervention’s funding, researchers also 
prefer those contracts which assure their own project’s financial stability. Researchers may 
want a year to do formative research, another year to do an intervention, and six months to 
do analysis and write up and distribute results. Look for ways to share early data quickly, and 
to draw interim conclusions. CBO staff can help suggest ways to do this through the data 
cleaning and analysis phases of the research project. This epidemic is still relatively young and 
changing quickly. A study can grow to be irrelevant if it takes too many years to complete. 


Disseminate negative results 

Researchers are also nervous about how results come out. Along with hoping the research 
will benefit the community, they also want to contribute to the scientific literature. For a 
variety of reasons, however, many researchers don’t publish articles when an intervention 
fails to show effective results. Not publishing articles about ineffective programs does a 
disservice to those who need to know the outcome of the research project so as to avoid 
repeating errors. If an academic journal won’t accept an article about unsuccessful programs, 
try to publish the results through conferences or self published reports. 


The conclusion of a collaborative research project is not really the end, but rather is part of 
a cycle. The only way for both researchers and CBOs to really fail is to pay no attention to 
the research findings. Research can help give the community at risk the services it deserves 
and needs, and collaborative research is an important tool in achieving that. Feel free to 
challenge the research, but let it also challenge you. 


KEY POINTS: INCORPORATING YOUR FINDINGS 


¢ Research findings are not used to judge goodness or badness of service provision, 
but to help improve prevention services. 


¢ Findings should inform both the researcher’s research direction as well as the 
CBO’s program direction. 


¢ Collaborative research findings can be used to start, modify or close programs. 
¢ Begin discussing potential implications of preliminary findings. 


¢ It is important to publish and disseminate findings about programs or methods 
that are ineffective. 


¢ Researchers and CBOs together can illuminate the findings through discussion. 


¢ The only way for a collaborative research project to fail is to pay no attention 
to the research findings. 
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Disseminating 
What You’ve Learned 


Without dissemination, we’re bound to repeat each other’s mistakes. 


Getting the word out should not be a last minute thought, but an integral part 


of a research plan. 


Chapter 


etting the word out about what the collaboration team has accomplished is 

crucial. CBOs, researchers, funders, health departments and Community Planning 

Groups are all eager to know more about what is happening and what is making a 
difference in HIV prevention. Too often, however, dissemination of research findings is 
considered only after the collaborative research is finished. For CBO staff, this might mean 
hashing out a report to the funder trying to justify the grant money. To a researcher, this 
might mean trying to find an extra few months to write and submit a journal article, or 
giving a formal presentation at a conference. 

One simple way to avoid this frustration is to build in a dissemination plan from the 
beginning of the collaborative project. Along with planning how the research question is 
formed, who conducts interviews and how the data is analyzed, be sure to plan who you 
want to reach with your findings, how to reach them, who on your team will write it up and 
how it gets delivered. 

If the research team has followed all the suggestions thus far in the manual, 
they will be ahead of the game for dissemination because they will have 
disseminated along the way. Meeting regularly and including a variety 
of CBO and research staff in meetings is one way to disseminate 
within the collaborative organizations. When inviting the head of 
the CBO or the project officer from the funding organization to a 
meeting, listen to their questions to get an idea of what is important 
for them to know. In addition to helping formulate the subject of 
the dissemination plan, this builds an audience for the final product 
by involving them in the process. 

Similarly, disseminate to persons outside of the collaboration as research 
progresses. For example, as part of the data analysis you might want to hold an informal 
focus group with clients or community members to see if they agree with the findings. 
While in data analysis, present findings to other researchers to help focus thoughts and get 
suggestions for dissemination through journals. Be sure all articles and presentations are peer 


reviewed by both researchers and CBO staff. 


MULTIPLE OUTLETS 


Dissemination can take any number of forms, from a conversation with a colleague in 
the hallway, to an article in an international journal. Different outlets have different 
audiences, from very specific to broad reaching. Remember that each outlet has a very 
different timeline, and time your dissemination plan accordingly. Also, the language and 


content for each can vary widely. 


Scientific journal articles — These are submitted and then peer reviewed before publishing. 
Can take anywhere from 6 months to a year before publication. Audiences are researchers, 


funders, media. 
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Magazine articles — These can be written and submitted or suggested via a query letter. 
Sometimes the magazine may decide to assign your article suggestion to a staff writer who then 
interviews you and writes the article. Timeline is 2-6 months. Audience is the general public. 


Newsletter articles — Can be an agency newsletter or wider community newsletter. Usually 
newsletters are eager to receive submissions. Timeline can be fast if newsletter is published 
monthly. Audience can be very targeted, usually CBOs, funders, activists. 


Reports — Self published reports can be mailed or faxed to any number of audiences. 
Timeline can be fast, depending on when you decide to publish. 


Press releases — Should primarily be considered only if the CBO or research institute has a 
media relations department. Research findings must be new, different and exciting to draw 
the interest of the press. Should get response within days, if any paper, television or radio 
station decides to write a story. Audience is the general public. 


Briefings — A good method for providing feedback to a targeted audience. Should be done 
with adequate outreach and groundwork so that you don’t throw a party and no one comes. 
Timeline is 4-6 weeks. 


Letters to the editor — This is a good choice for presenting your research while voicing an 
opinion. Can appear in a few days if accepted. Audience is the general public. 


Presentations, workshops at conferences — One of the best ways to present research findings 
in a personal way. Also provides for instant feedback and discussion. Abstracts for oral 
presentations or posters are submitted and reviewed for acceptance. Timeline is 4-6 months. 
Conferences have different audiences, so be sure to choose those that target both 


researchers and CBOs. 


Internet — Posting information on a web site is fast and relatively easy. If you don’t have 
your own site, look to a reputable web page in your field. Timeline is only a few days, but 
the audience is highly unknown, ranging from highly targeted to very general. 


MULTIPLE AUDIENCES 

Along with multiple outlets, there are multiple audiences for disseminating information. 

A dissemination plan should take into account all the different persons who might benefit 
from hearing about the research project. Collaborative partners should share mailing lists to 
reach the broadest possible audience. 


CBO — Invite not only the director, but the outreach staff and everyone involved in 
providing services. Outlets: report, briefing, newsletter. 


Research institute — Often academic institutes have Brown Bags or other seminars built 
into their schedules. Outlets: report, briefing, journal article. 


Clients — If you’ve interviewed clients, be sure to let them know what you’ve learned. 
Outlets: flyer, newsletter, briefing, Internet. 
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Other CBOs — Other agencies working with similar populations can benefit from what 
’ * 
you've learned. Outlets: report, briefing, newsletter, conference, Internet. 


Other researchers — Other scientists will be interested in your findings. Outlets: journal 
article, conference, Internet, press release. 


Community groups — Your research could be of interest to community groups outside of 
HIV prevention, such as family planning clinics, STD clinics, violence prevention groups, 
Alcoholics Anonymous or Narcotics Anonymous, boys or girls clubs, etc. Outlets: report, 
magazine article, newsletter, press release, letter to the editor, Internet. 


Department of Public Health and Community Planning Groups — These two groups, who 
are responsible for many funding decisions, should always receive research summaries. 
Outlets: report, briefing, journal article, magazine, newsletter, Internet. 


Key decision makers — Depending on your study, other influential groups in your 
community might like to hear about your work: law enforcement officials, school officials, 
local politicians, etc. Outlets: report, magazine, press release, letter to the editor, Internet. 


BRIEFINGS 
Briefings are one of the most popular and effective ways of disseminating information. 
Unless there is a built-in audience (clients at the intervention, agency staff at an in-service, 
etc.), don’t just assume that people will show up. Get the invitation out in time for people 
to put it on their calendars (2-4 weeks in advance) and follow-up a day or two before to get 
RSVPs. If a variety of people have been involved from the beginning, they are more likely 
to want to come to a briefing at the end of the research project. 

Here are some tips for planning an interesting and dynamic briefing: 


© Summarize findings. Break the results down into easily understood categories. Turn 
each concept into a 3 or 4 word phrase. Leave details to handouts. 


e Find the best presenter. Presenters should be dynamic and good speakers. Include a 
mix of CBO staff, outreach workers and researchers. 


* Practice. Have each presenter practice in front of an audience before the briefing to 
get feedback and increase confidence. 7 


° Use visual aids. Slides, overheads, PowerPoint presentations all break up the talk and 
give the audience something to focus on. 


e Make it personal. Whenever possible, include quotes from interviews, personal stories 
or photos of the intervention. Bring in one of the clients and let them tell their own 
story. Remember to get written permission before using any photographs of clients and 
to protect clients’ identities in any materials presented. 


* Provide food. Any enticement will increase the turnout for your presentation. Once 
people get there, they will be interested to hear what you have to say. 


Chapter 7: Disseminating What You've Learned | Page 53 


© Provide written handouts of everything covered. Every slide and overhead should be 
copied for handouts. Outlines of each person’s talk should also be available to take 
home. You can provide additional materials that weren’t covered as well. 


° Leave time for interaction/questions. Plan for at least 20 minutes of questions at the 
end of the briefing. This is the time for you to hear reactions and clarify anything that 


might have been omitted. 


e Give contact information. Include contact information on all handouts. Bring lots of 
business cards if you have them. Be sure to include name, address, phone, fax, e-mail 


and Internet site. 


Adapted from Making a Difference to Policies and Programs: A Guide for, Researchers. 


Without dissemination, we’re bound to repeat each other’s mistakes. Getting the word 
out should not be a last minute thought, but an integral part of a research plan. 


KEY POINTS: DISSEMINATING WHAT YOU"’VE LEARNED 


All of the dissemination strategies listed above can be done collaboratively. Although you 
may not want to do every single dissemination activity together, CBO staff and researchers 
should decide together who does what. To make the process truly collaborative, here are 
some suggestions: 


¢ When giving briefings, have both researcher and CBO staff present. It is especially 
important to invite the CBO staff to present at the research institution. Likewise, 
researchers should present along with the CBO staff at agencies. 


¢ For published materials, include CBO staff name on journal articles, researcher 
name in newsletter articles. Allow CBO or researcher partner to have significant 
input into the articles. 


e Apply for a presentation or workshop at a conference and give it together. Researchers 
should invite CBO staff to present with them at traditionally research-oriented 
conferences. CBO staff should encourage researchers to present at traditionally service- 
oriented conferences. 


¢ Help recruit the audience for briefings. Use combined professional networks to get the 
word out. Share mail, e-mail and phone lists whenever feasible. 


¢ Train each other. If CBO folk have never written journal articles, take the time to give 
them examples to read and guidelines on what to expect. Likewise, if a researcher has 
never presented to an audience of service providers, brief them on what to expect and 
help shape the presentation. 


* Include CBO staff as peer reviewers for research articles, grant proposals and presentations. 
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Cast STUDY 


NEW CONSERVATORY THEATER CENTER 
% 
by Lisa Heft 


ew Conservatory Theatre Center is an agency that develops 


interactive theatre-in-education touring programs on social 


issues for young people. We decided to evaluate the effec- 
tiveness of one of its HIV/AIDS plays for high school youth as a means 


for increasing parent-adolescent communication on the subject. 


FLEXIBLE TIME 

Luckily the funding covered a half-time 
position for me. | cannot imagine adding 
this research project to an existing workload 
without full agency support and their real- 
ization that this would be a time-intensive 
project. When things were slow I was able to 
do some grant writing for the agency. During 
busier phases in the research project I was on 
the road conducting research or sitting in a 
little room for hours counting, cleaning and 
sorting completed surveys preparing them for 
data entry. 


AN ARRANGED MARRIAGE, BUT A GOOD ONE 

I had the wonderful good fortune to be 
matched with Tom Hall, a researcher whose 
style fit mine, who had an interest in 
theatre, and who treated me as a fellow 
expert and colleague. 


AWASH IN A SEA OF PAPER 

[had a tiny office with a desk and a phone to 
do my work, and if you were to look in the 
door you would maybe find my little face 
peeking out amidst the towers of 21 stacked 
boxes and two file cabinets full of surveys. 
During one particularly rainy season our 


Case Study: New Conservatory Theater Center 


basement offices flooded from the street above 
— if you were to look in the door at that time 
you would have seen my little face poking out 
between towers of waterproof tarps. 


ONE DAY MY JOB CHANGED 

At one point I had to take a full-time job at 
another agency. What would happen with 
my research project? Would this be the 
obstacle that prevented its success? I made a 


Teen actors in a New Conservatory Theater Center 
production at a high school. 
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deal with my new workplace and was able to 
move my project there, working on it after 
hours. Tom coming to my worksite gave the 
project credibility with my new supervisors. 


IT WANTS MONEY 

We had to do an unforeseen mailing of over 
3,000 consent forms. | had to hire a temporary 
worker to help count and code the forms 
with me during a time when I was needed in 
my day job. Once I had to pay someone to 
drive the surveys up to the study community. 
When my abstract was accepted at a confer- 
ence | had to print up a billboard-sized poster 
to explain the research. Little money surprises 
throughout. Fortunately the funder was 
involved and flexible and supported these 
additional costs. 


WHAT IF NOBODY COMES? 

Our hypothesis was that giving the same 
dramatic intervention to parents and their 
teenagers would increase intergenerational 
communication on HIV/AIDS. But so few 
parents came to the performances that we did 
not have enough to even study. That might 
have been the demise of the whole project. 
Luckily, though, we included an optional 
essay question: “Do you have anything to say 
about increasing parent-teen communica- 
tion about HIV/AIDS?” And though young 
people may seem to be apathetic at best 
about information coming from their 
parents, we received over 1,200 responses 


Page 56 


. 


overwhelmingly asking us to help parents 
talk more with their kids. So we clustered, 
coded and counted the responses and the 
wealth of information this represented was 
so rich we are still using it years later. And 
the fact that nobody showed up for the 
parent performances was in itself useful and 
concrete data. 


HOW DID IT ALL TURN OUT? 

With help from the collaborators I was able 
to conduct the research and write a paper. 
It was posted on the research agency’s web 
site, and they helped me submit it to the 
International AIDS Conference, where it 
was accepted for a poster presentation. My 
researchers and funders found extra money 
to send me to my first conference ever! I also 
presented the findings locally to other scien- 
tists and community researchers in briefing 
sessions and at a national conference. And I 
was invited to help write an internationally 
distributed fact 
communication. 


sheet on parent-teen 


It was hard work, but my scientist and 
funder partners kept me inspired and headed 
in the right direction. The experience was 
marvelous, and I learned many things — 
from evaluation methods to writing research 
papers to how to present at a conference. 
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Appendix The CAPS/NCG Collaborative Research Consortium Model 


is 
CAPS has developed and evaluated a successful model for research consortia that has been replicated 
across the country. The following were the key components of the model: 


e A consortium of 11 CBOs, matched researchers, 2 statisticians, a program manager, and 
administrative support staff 


© 10-20% of a CAPS researcher’s time per research project, with an additional dedicated 
statistician for each team 


© Up to $50,000 in intervention and $10,000 evaluation funding per year for each CBO 


e An initial 4-day workshop which provided CBO participants extensive training in research 
design, data collection and analysis and allowed the research/CBO teams an opportunity to 
begin crafting their project 


© Monthly meetings where participants received additional training, collective problem solving 
and support 


e A working partnership with local corporate and private philanthropic funders (Northern 
California Grantmakers — AIDS Task Force) who actively participated in monthly meetings 
and problem-solving 


e A program manager to support the scientist-CBO pairs, facilitate training and assistance and 
remove obstacles to research project implementation 


e Ongoing technical assistance, including scientist and statistical consultation, data entry and 
statistical analysis 


e Additional funding to disseminate findings to colleagues and community members 


THE MODEL: 
© The project was initiated by a Request for Proposals (RFP) issued by the funder. 


e After the agencies were selected, the University hosted an intensive workshop course on the 
basics of research and evaluation. The workshop was conducted over four consecutive weeks, 
for a total of 32 hours of instruction. Topics included protocol design sample selection, instru- 
ment development, staff training, research implementation, and data cleaning and analysis. 
The small group format provided the CBO staff the opportunity to develop peer support for the 
learning they were doing. 


¢ Following the workshop, CBOs were each paired with a CAPS researcher based on common 
expertise in population, the needs of the research or proximity. Each team negotiated the final 
details of the research question. They also negotiated their respective roles and a schedule for 
staying in contact. These varied according to the needs of the participants, but an initial meet- 
ing at the CBO to meet the CBO staff was standard across all projects. 


© The entire group (CBO staff, researchers, funders and consortium management and support) met 
for a two-hour lunch meeting monthly for updates and training. 


© Researchers and CBOs jointly developed the research instruments, researchers completed the 
Committee on Human Research (CHR) applications, and CBOs were responsible for data collec- 
tion. Researchers participated in quality control of data collection, and several participated in 
staff training at the CBO, as well. Data analysis was conducted jointly, as was the dissemination. 
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We recognize that this fairly resource-intensive, “Cadillac” version of collaborative research will 
not be accessible, available, or even appropriate for everyone in all situations. The added value of the 
consortium was significant. 


Projects were able to learn from each other. 


CBOs were able to teach, learn and get support from other CBOs also trying to negotiate 
relationships with researchers. 


Academic researchers also got to teach, learn and get support from other researchers about 
negotiating relationships with CBOs and conducting community-based studies. 


Because the group met monthly, the consortium provided a convenient, regular means for 
additional training. 


Program manager and funder were able to track progress and problems in multiple collaborative 
projects and intervene with assistance when warranted. 


Page 60 Appendix B 


Appendix Finding Collaborative Research Partners 


GRADUATE SCHOOLS OF PUBLIC HEALTH, PSYCHOLOGY, ETC. (finding researchers) 

Most Master's programs for health-related subjects require students to work in a service organization 
as part of their studies. This is a great way to get a junior researcher for free. It’s also a good way to 
start a relationship with a researcher who is just starting out, and will hopefully continue the process 
of collaborative work in the future. Junior researchers may be more likely to work within the service 
organization, spending time at the agency to see how services work. 

Graduate student researchers may have limited time and expertise, so be sure to keep realistic 
expectations for the collaboration. For example, a graduate student researcher may not have expert 
SPSS skills, so they may need to work with an advisor to conduct a detailed data analysis. 

The Association of Schools of Public Health lists all the accredited schools on its web page: 
http://www.asph.org/aa_section.cfm/151. 


STATE AND LOCAL HEALTH DEPARTMENTS (finding researchers and service providers) 
All service agencies that are funded by state or local health departments are entitled to technical assis- 
tance (TA). Many researchers at health departments are already collaborating with service agencies. 
Also, researchers at health departments are familiar with service agencies and can help outside 
researchers hook up with agencies that may need their services. Collaborations with health departments 
are likely to be more long-term, as the relationship will continue through funding and other sources. 
The NASTAD web site has a list of names and contacts for state health departments: 
http://nastad.vertex.net/res_state_directory.asp ’menu=res. 


CDC (finding researchers and service providers) 

Again, agencies that are funded by the CDC are entitled to TA, including capacity building and 
evaluation. Contact your program officer for more information. The CDC’s National Prevention 
Information Network (NPIN) maintains a searchable database of agencies working in HIV 
prevention. Researchers can find agencies working in their community by calling the hotline 


(800-458-5231, 800-243-7012 TTY) or going on-line: http://www.cdenpin.org/db/public/rsmain.htm. 


BEHAVIORAL AND SOCIAL SCIENCE VOLUNTEER PROGRAM (BSSVP) 

(finding researchers and service providers) 

The American Psychological Association, Office on AIDS established the BSSVP to help researchers 
on the local level get involved with community-based service agencies. The BSSVP links researchers 
and service providers for collaborations. The researchers are volunteers, and the program trains 
researchers on collaborative work, making them more likely to be prepared for the special needs of 


collaboration. 


Contact: E. Duane Wilkerson, M.P.H., Program Director 
(877) 754-1404 
DWilkerson10@home.com 


Contact: Robin Kelley, Program Coordinator 
RKelley@apa.org 
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Appendix Glossary of Research Terms 


CLOSED-ENDED QUESTIONS: Questions in an interview or survey format that provide a limited set of 
predefined responses. 


COMPARISON GROUP: A group of individuals who have similar characteristics to the program participants 
(such as race/ethnicity, gender and age) which is compared to the program (or intervention) group in 
an evaluation process to determine which type of services, activities or products provided by your 
program produced the expected changes. The comparison group may not receive any services or may 
receive a different set of services, activities or products. 


CASE sTuDY: A non-experimental design that provides a detailed analysis of an individual program. 
The focus is on description of activities, not on determining cause. Case studies may be performed on 
several programs simultaneously to allow qualitative comparisons of certain aspects of the programs. 


COHORT STUDY: See Longitudinal Study. 


CONTROL GROUP: A group of individuals whose characteristics (such as race/ethnicity, gender and age) are 
similar to the program/intervention participants but do not receive the program or intervention evalu- 
ated. This is similar to a comparison group, except that participants are randomly assigned either to the 
intervention group or the control group. A control group is used to assess the effect of your program on 
participants as compared to similar individuals not receiving the programs or activities evaluated. The 
same information is collected for people in the control group as in the experimental group. 


CORRELATION: A statistical measure of the degree of relationship or association between variables. 


CROSS-SECTIONAL STUDY: A cross-section is a sample of a population. A cross-sectional study examines 
this sample at one point in time. Successive cross-sectional studies can be used as a substitute for a 
longitudinal study. 


DATA: Specific information or facts that are collected. Variables can be demographic, behavioral, 
descriptive, etc. 


DATA ANALYSIS: The process of systematically applying certain techniques (e.g., statistics) to describe, 
summarize and compare the data collected. 


FORMATIVE EVALUATION: Evaluation during the design and pre-testing of programs to guide program 
development. Used to assist planners, managers and staff to develop a new program or improve an 
ongoing program. 


IMPACT EVALUATION: The assessment of the effects of an intervention beyond the outcomes on individ- 
uals targeted by the intervention, i.e., what is the impact of the availability of the program on the 
broader community? 


INCIDENCE: The number of new cases of a disease that occurs in a specified population during a 
specified time period. 


INTERVENTION: A specific activity or set of related activities intended to bring about HIV risk reduction 
in a particular target population using a common strategy. An intervention has distinct process and 
outcome objectives and a protocol outlining the steps for implementation. 


LONGITUDINAL STUDY (also called COHORT STUDY): A study in which the same persons are observed repeat- 
edly over time, usually at regular intervals. 


NEEDS ASSESSMENT: The process of obtaining and analyzing information from a variety of sources in 
order to determine the needs of a particular client, population or community. 


OPEN-ENDED QUESTIONS: Questions in an interview or survey that allow the respondents to provide 
their own answers, as. opposed to having to select one of a limited set of predefined responses. 
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OUTCOME EVALUATION: The process of determining the degree to which a program or intervention 
achieves the intended objectives. Outcome evaluation is concerned with detecting any changes as a 
result of the intervention, and the degree to which they are the result of the intervention. 


OUTCOME OBJECTIVES: The overall intended effects of the intervention, specifying its purpose and 
mission. These might includeincreased knowledge about HIV, changing risk-related behaviors, 
promoting community norms for safer sex or reducing HIV transmission. 


PILOT TEST: A trial run with a few participants to assess the appropriateness and feasibility of the 
intervention procedures and data collection instruments such as questionnaires. 


PRE AND POST DESIGN: An evaluation design in which a single program or group is measured prior to 
program implementation and again after the end of the program. 


PROCESS EVALUATION: A descriptive assessment of the implementation of program activities, what was 
done, to whom, how, when and where, to learn how the program was implemented and if it reached 
its intended audience. 


PROCESS OBJECTIVES: These describe the specific intervention activities, what is needed to carry them 
out, the people responsible for carrying them out, and when they will be completed. 


QUALITATIVE DaTA: Data that is in narrative or text form, for example, information collected from 
focus groups or in-depth interviews. 


QUANTITATIVE DATA: Data that is numerical, in the form of numbers, for example, survey data and data 
from epidemiological reports. 


RANDOM ASSIGNMENT (or RANDOMIZATION): A method of assigning research participants to two or more 
groups by chance. Randomization is done to draw samples that are representative of a known popula- 
tion and to ensure that groups can be statistically compared. 


RANDOM SAMPLING: A technique where each research participant is chosen entirely by chance and 
each member of the population has an equal chance of being selected for the sample. 


RELIABILITY: The consistency of a measure or a question in obtaining very similar or identical results 
when used repeatedly. 


REPRESENTATIVE: Indicates that a sample is similar to the population from which it was drawn and, 
therefore, can be used to draw conclusions about that population. 


RESEARCH DESIGN: A plan that defines the number and type of variables to be studied and assesses their 
relationship to one another using well-developed principles of scientific inquiry. 


SAMPLE: A group of participants selected from a total population with the expectation that studying 
the group will provide important information about the total population. 


STATISTICAL SIGNIFICANCE: A measure of whether an observed difference or relationship is larger or 
smaller than would be expected to occur by chance alone. Statistical results are often considered to 
be significant if there is less than a 5% chance (.05 or 5 out of 100) that they would have occurred 
by chance alone. 


TARGET POPULATIONS: Groups of people who are to be reached through the intervention. 


VALIDITY: The extent to which a survey question or other measurement instrument actually measures 
what it is supposed to measure. 


VARIABLE: A characteristic that can change or vary among different people or in the same person over 
time. For example, race or ethnicity varies among individuals, and income varies for the same indi- 
vidual over time. 


Adapted from Evaluating CDC-Funded Health Department HIV Prevention Programs, Glossary of Terms 
http://www.cde.gov/hiv/aboutdhap/perb/TERMSfinal. pdf 
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Appendix Other Manuals 


Good Questions/Better Answers: A Formative Research Handbook 
for California HIV Prevention Programs 

Center for AIDS Prevention Studies 
http://www.caps.ucsf.edu/goodquestions/ 


A Bridge Between Communities (videotape) 

The Detroit Community-Academic Urban Research Center 
University of Michigan School of Public Health wii 
Phone (734) 764-5171 


rojmcg@umich.edu 


Getting Started: The NMHA Guide to Establishing Community-Based PreventionPrograms 
National Mental Health Association 

1021 Prince Street 

Alexandria, VA 22314-2971 

Phone (703) 684-7722 

Fax (703) 684-5968 


The Program Development Puzzle: How to Make the Pieces Fit 
National Minority AIDS Council 
http://www.nmac.org/NewsLetters/NMAC/Technical/puzzle.pdf 


Community-Based AIDS Prevention and Care in Africa: Building on Local Initiatives 
Population Council 
http://www.popcouncil.org/ebert/wellbriefing.html 


Knowledge from Action: Community-Based Research in Canada’s HIV Strategy 
Canadian HIV/AIDS Clearinghouse 

Canadian Public Health Association 

1565 Carling Avenue, Suite 400 

Ottawa, Ontario, Canada K1Z 8R1 

Phone (613) 725-3434 

Fax (613) 725-1205 

aidssida@cpha.ca 


Making a Difference to Policies and Programs: A Guide for Researchers 
The United States Agency for International Development 

Health and Human Resources Analysis for Africa 
http://www.usaid.gov/regions/aft/hhraa/guide/eng_intro.htm 


The California Wellness Foundation: Reflections on Capacity Building 
California Wellness Foundation 


http://www.tcwf.org/reflections/2001/april/index.htm 
Beyond Our Borders: A Guide to Twinning for HIV/AIDS Organizations 


Interagency Coalition on AIDS and Development 
http://www.twinningforaids.com/ 
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